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Executive Summary 



The Rural and Low-Income School (RLIS) program is part of the Rural Education Achievement 
Program (REAP) that was authorized under Title VI, Part B of the Elementary and Secondary 
Act of 1965 (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). The REIS 
program provides additional funds to help rural districts serving low-income students make 
adequate yearly progress (AYP) as described in Section 111 1(b)(2) of the ESEA. REIS funds 
may be used to support a variety of activities, including teacher recruitment and retention; 
teacher professional development; support for educational technology; parental involvement 
activities; activities authorized under the Safe and Drug-Eree Schools Program; and activities 
authorized under Title I, Part A, and Title III of the ESEA. 

REIS funds are distributed to state education agencies, which then distribute money to the school 
districts that meet the following criteria: (a) the district is not eligible for a grant from the Small, 
Rural School Achievement (SRSA) program, which serves rural school districts that have fewer 
than 600 students or that serve extremely sparsely populated areas; (b) 20 percent or more of the 
children ages 5 through 17 served by the district are from families with incomes below the 
poverty line; and (c) all of the schools included in the district must have a National Center for 
Education Statistics (NCES) locale code of 6 (small town), 7 (rural), or 8 (rural near an urban 
area). The REIS program distributed almost $86 million to 41 states in the 2009-10 school year. 
In turn, the states distributed REIS funds to 1,497 districts. Award amounts in 2009-10 averaged 
approximately $57,000 per district and $29 per pupil in REIS-funded districts. 

This report includes findings from interview and survey data obtained from state and district 
REIS coordinators who were selected for the interview or survey samples based on their state or 
district’s receipt of REIS funds for the 2007-08 school year. It describes state and district 
implementation of the REIS program, priorities for REIS funds, and REIS district characteristics. 
It also includes results from an analysis of extant data at the state and district levels on student 
achievement outcomes from state assessments used for NCLB accountability. 

Key Findings 

• In addition to the primary goal of making AYP, district and state survey respondents 
reported that REIS funds were primarily used to purchase educational technology, 
support teacher professional development, and, in general, support activities authorized 
under Title I, Part A. Similarly, interviews with district REIS coordinators and reviews of 
district documents indicated that districts primarily used REIS funds for technology 
hardware and software, teacher pay, and professional development. 

• This evaluation did not examine causality and achievement gains cannot be attributed to 
the REIS program; however, from the 2002-03 school year to the 2007-08 school year, 
the rate of academic improvement in mathematics and reading for districts that received 
REIS funding was significantly greater than for non-REIS rural districts. There was no 
systematic relationship between the number of years a district received REIS and gains in 
student achievement. 
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State and District Implementation of the RLIS Program 

To understand how states and school districts use RLIS funds, it is necessary to look first at the 
goals and priorities the states and the RLIS districts establish for the program, and how they are 
aligned with overall district needs. The most common goals for district RLIS programs were 
improving the quality of instruction and improving student achievement in specific subject areas. 
All nine states in the state interview sample required RLIS districts to engage in a comprehensive 
planning process to identify local needs, and the majority of district coordinators in the district 
interview sample reported undertaking such a process to set the goals and priorities for their 
districts. However, interview and document data showed considerable variation among states and 
districts with regard to the planning tools they used and the extent to which districts were 
directed through the planning process. 

State and district RLIS staff reported that planning and administrative processes for the RLIS 
program are, in virtually all instances, integrated into a consolidated planning and application 
process for federal programs that requires districts to show how they will use the different 
funding sources to address identified needs and meet student achievement goals. All of the 
interviewed states provide a considerable amount of training and technical assistance to the 
districts to provide them with information about the RLIS Program and assist them with the 
application process. In particular, the states provide the districts with assistance in how to 
identify their specific needs for improvement and how to focus their use of RLIS funds in ways 
that support their efforts to improve student achievement outcomes in reading and mathematics 
and achieve their AYP goals. District coordinators reported receiving regular, knowledgeable 
and helpful assistance from state RLIS coordinators in developing their applications for RLIS 
funds. 

RLIS coordinators reported that the flexibility of the RLIS program allowed them to use these 
funds to meet specific needs in their districts. Interviews with these district RLIS coordinators, 
along with reviews of RLIS-related documents and technical assistance materials obtained from 
state and district interview respondents, indicated that districts primarily used RLIS funds for 
teacher pay, educational technology, professional development, and materials. In an online 
survey of all states and a random sample of districts that received RLIS funds, both state and 
district survey respondents reported that RLIS funds were used to support activities authorized 
under Title I, Part A, and to purchase educational technology. 
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District Characteristics 



In addition to investigating the implementation process for the RLIS program, this evaluation 
provides information on the demographic characteristics of eligible RLIS districts, and compares 
those characteristics to those of the national average and the average of rural districts that did not 
qualify for RLIS funds. These analyses found that the characteristics of students in RLIS districts 
remained relatively stable in the four school years examined, 2003-04 to 2006-07. These 
characteristics include the following: 

• On average, RLIS districts had more students than other rural districts and fewer students 
than all districts nationally. 

• RLIS districts were more concentrated in the South than other rural districts and districts 
nationally. 

• Student-teacher ratios in RLIS districts were slightly lower than in other districts 
nationally, but slightly higher than in other rural districts. 

• Total per-pupil spending in RLIS districts slightly increased between 2003-04 and 
2006-07, from $8,478 to $9,842 (dollars unadjusted for inflation). However, per-pupil 
spending remained substantially lower in RLIS districts than in all districts nationally or 
in other rural districts, indicating the RLIS program is targeting districts as intended. 

• Compared with districts nationally, students in RLIS districts were more likely to be 
white, black, American Indian or Alaskan Native and less likely to be Hispanic, Asian or 
Pacific Islander. Compared with other rural districts, students in RLIS districts were less 
likely to be white and more likely to be black or Hispanic. 

• RLIS districts served a higher proportion of students who qualified for free or reduced- 
price meals and a slightly higher proportion of students who had an Individualized 
Education Program (lEP) compared with districts nationally and non-REIS rural districts. 

• Students in REIS districts were less likely to be limited English proficient than students 
nationally but more likely to be limited English proficient than students in non-REIS 
rural districts. 

Student Achievement 

Increasing student achievement and assisting districts in making AYP are primary goals of the 
REIS program. To assess the efforts of districts across the country toward achieving these goals, 
the evaluation examined student achievement in REIS districts, first, by performing a descriptive 
analysis of how REIS districts were faring under the AYP provisions of NCLB in the 2007-08 
school year; next, by examining trends in student achievement from 2002-03 through 2007-08; 
and finally, by examining the relationship between student achievement and selected 
characteristics of REIS implementation in a sample of REIS districts. 
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Overall, 54 percent of RLIS districts met all AYP targets in mathematics and reading in 2007- 
OS. Among racial and ethnic subgroups, RLIS districts had the most difficulty making AYP for 
the black student subgroup, although the majority of RLIS districts with the minimum number of 
students made AYP for this group. ^ Among all subgroups, RLIS districts had the most trouble 
making AYP for students with lEPs. 

This evaluation did not examine causality and achievement gains cannot be attributed to the 
RLIS program; however, the rate of academic improvement in mathematics and reading for 
districts that received RLIS funding was significantly greater than for non-RLIS rural districts 
between 2002-03 and 2007-08. While the differences in growth are statistically significant, it is 
important to note that they are relatively small in size. Annual growth in RLIS districts was 0.03 
standardized units higher in mathematics achievement and 0.02 standardized units higher in 
reading achievement than in non-RLIS district controlling for several district characteristics. 
There was no systematic relationship between the number of years a district received RLIS and 
gains in student achievement. All RLIS groups — whether receiving one, two, three, four, five, or 
six years of RLIS funding — exhibited positive gains relative to non-RLIS districts though not all 
of these gains were statistically significant. 

Exploratory analyses suggest that districts that set REIS goals to decrease their dropout rates and 
increase the quality of instruction had significantly higher mathematics achievement than 
districts that did not set these goals. However, there was no relationship between a reported focus 
on specific activities and mathematics or reading achievement. 

Sources of Data 

The evaluation’s findings are based on multiple sources of data. To answer the questions on 
REIS program administration, REIS funding priorities for the districts and states, and technical 
assistance provided to the states by the districts, the evaluation included in-depth interviews and 
an analysis of extant state documents in nine of the 39 states that received REIS funding for the 

2007- 08 school year and in 43 of 45 randomly selected districts (five districts in each of the nine 
sampled states, with two non-responses) that received REIS funding in 2007-08. The state 
interviews were conducted by telephone during the spring of the 2007-08 school year and the 
district interviews were conducted by telephone during the late spring-early summer of the 

2008- 09 school year. The nine states in the sample were those receiving the largest REIS 
allocations in 2007-08: It is important to note that these nine states were selected to provide 
qualitative information about state priorities, program administration, and technical assistance, 
and were not intended to represent state implementation across the 39 states receiving funding. 

REIS program data files available on the Department’s Web site were used to identify which 
districts were eligible for REIS funds. In 28 of the 43 states that had ever received REIS funding, 
all REIS-eligible districts received funding in each year they were eligible. Additionally, the 
Common Core of Data (CCD), a federal program that annually collects data about all public 



* Each state sets its own minimum n-size for subgroup reporting. 
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schools, public school districts and state education agencies in the United States, provided 
district demographic information. 

The evaluation’s longitudinal analysis used data from the EDEN-EDFaets data system, which 
contains student aehievement and accountability data mandated in NCLB. It therefore includes 
data on student achievement on state assessments and data on AYP for all districts in the U.S. In 
2009, the EDEN-EDFacts data system included achievement data from the 2005-06, 2006-07, 
and 2007-08 sehool years. To supplement the EDEN-EDFacts achievement data, the evaluation 
drew on data colleeted in the National Eongitudinal Sehool-Eevel Assessment Seore Database 
(NESESASD) to gather district achievement data from the 2002-03 through 2004-05 school 
years. NESESASD was a federal effort, which predated the EDEN-EDFacts data system, to 
eolleet and refine data from state assessment systems. The CCD data were also used as 
eovariates in modeling the assoeiation of receipt of REIS funds and student achievement. In both 
the EDEN-EDFacts and NESESASD data, the grades assessed and the assessments used vary by 
state. The analysis used data from all grades in which there were assessment data for all students 
in reading or mathematics. To account for the faet that the assessments are not comparable from 
state to state and the grades varied by state, profieieney levels were eon verted to z-seores to 
assess the relative ranking of REIS districts within their state. In addition, both data sources 
report percent proficient to measure achievement rather than scale scores on an assessment. As a 
result, the analyses only provide information about aehievement around the proficiency cut point, 
but do not provide information about ehanges in performanee among high- or low-performing 
students. 
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Chapter 1: Introduction and Background 



The Rural and Low-Income School (RLIS) program is part of the Rural Education Achievement 
Program (REAP) that was authorized under Title VI, Part B, of the Elementary and Secondary 
Act of 1965 (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). Rural school 
districts with a high prevalence of students from low-income families in their communities often 
experience financial disadvantage due to a reduced property tax base, which is foundational in 
district funding. The REIS program provides additional funds to help rural districts serving low- 
income students make adequate yearly progress (AYP) as described in Section 111 1(b)(2) of 
ESEA. An additional grant program created under REAP, the Small, Rural School Achievement 
(SRSA) program, targets school districts that have fewer than 600 students or that serve 
extremely rural areas. SRSA funds are provided directly to eligible school districts by the U. S. 
Department of Education, while REIS funds are awarded to state education agencies which then 
distribute the funds to the school districts in their state that are determined by the Department to 
meet the REIS eligibility criteria. The REIS program distributed almost $86 million to 41 states 
in the 2009-10 school year. In turn, the states distributed REIS funds to 1,497 districts. Award 
amounts in 2009-10 averaged approximately $57,000 per district and $29 per pupil in REIS- 
funded districts. 

To be eligible for REIS funds, a district must show that: 

• The district is not eligible for an SRSA grant; 

• Twenty percent or more of the children ages 5 through 17 served by the district are from 
families with incomes below the poverty line; and 

• All of the schools included in the district have a National Center for Education Statistics 
(NCES) locale code of 6 (small town), 7 (rural), or 8 (rural near an urban area).^ 

REIS funds are meant to be flexible and can be used to support a variety of activities, including 
teacher recruitment and retention; teacher professional development; support for educational 
technology; parental involvement activities; activities authorized under the Safe and Drug-Eree 
Schools Program; and activities authorized under Title I, Part A, and Title III of ESEA. 

The findings of this study will provide information to the Department for REIS program 
management and improvement. Additionally, these findings will help the Department prepare 



^ To be eligible to receive SRSA program funds, a district must show that: (a) the district has a total average daily 
attendance of fewer than 600 students or serves only schools that are located in counties with a population density of 
fewer than 10 people per square mile; and (b) the district includes only schools that either have a National Center for 
Education Statistics (NCES) locale code of 7 (rural) or 8 (rural near an urban area) or are located in an area of the 
state defined as rural by a governmental agency of the state. 

^ See Office of Elementary and Secondary Education Web site. Rural Education Achievement Program (REAP), 
SRSA program eligibility (http://www.ed.gov/programs/reapsrsa/eligibility.html) and RLIS Program eligibility 
(http://www.ed.gov/programs/reaprlisp/eligibility.html). 
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congressionally mandated biennial reports on the RLIS program and its participation in required 
performance reporting, accountability, and program assessment activities. 

Scope of This Final Report 

Key purposes of this study are to identify how states and school districts use RLIS program 
funds, to assess the progress states and districts made toward RLIS program goals, and to gain 
insights into what factors acted as facilitators and barriers to meeting state RLIS goals. 

This final report contains findings based on the following data collections and analyses: 

• Extant data at the state and district levels on student achievement outcomes from state 
assessments used for NCLB accountability 

• Demographic data for districts eligible for RLIS funding 

• Telephone interviews with RLIS coordinators from a sample of nine state education 
agencies and 43 districts regarding RLIS goals, priorities, and uses of funds 

• RLIS-related documents, such as federal grant application instructions and technical 
assistance guides, obtained from the sampled states and districts 

• An online survey of staff members from all states receiving RLIS funding in 2007-08 

• An online survey of a random sample of RLIS coordinators from districts receiving RLIS 
funding in 2007-08 

• Summaries of monitoring visits conducted by the U.S. Department of Education with 
states receiving REIS funding 

Evaluation Questions 

This study addresses the following evaluation questions: 

1 . What are the characteristics of the districts served by the REIS program in terms of rural 
location, poverty, race, etc.? 

2. What are the achievement trends in REIS districts compared with other rural districts? 

3. What progress have states made toward achieving their REIS goals? 

4. What are states’ priorities for districts’ use of REIS grant funds? How do states 
administer and monitor the program? What guidance and assistance do states provide? 
How do states enforce the statutory accountability provisions? 

5. What goals have districts identified for REIS in their grant applications? What progress 
have districts made toward their goals? How have districts actually used REIS funds? 

The interim report for this study contained findings and analyses that addressed evaluation 
questions 1 and 4. This final report provides updated findings for questions 1 and 4 and findings 
for questions 2, 3, and 5. 
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Data Collection and Analysis 

State and District Interview and Document Data 

To answer the questions on RLIS program administration, RLIS funding priorities for the 
districts and states, and technical assistance provided by the states to the districts, the evaluation 
included in-depth interviews and an analysis of extant state documents in nine of the 39 states 
that received RLIS funding during the 2007-08 school year (the state interviews were conducted 
by telephone in the spring of the 2007-08 school year) and in 43 of 45 randomly selected 
districts that received RLIS funding in 2007-08 in those nine states. The district interviews were 
conducted during the late spring-early summer of the 2008-09 school year. The interview guides 
that were used for the state and district interviews are in Appendices C and D. 

The state interviews were conducted with the RLIS coordinators in the nine states that received 
the largest RLIS allocations for the 2007-08 school year (see Exhibit 1). Together, these states 
accounted for 62.4 percent of RLIS funding in 2007-08."^ 



Exhibit 1 

Rurai and Low-Income Schooi Funding for the Nine Sampie States 



State 


2007-08 Funding 


Total Funding From 
the RLIS Program 
(2002-03 Through 
2008-09) 


Texas 


$7,512,087 


$53,231,150 


Georgia 


$7,258,669 


$51,295,765 


Mississippi 


$7,132,600 


$44,797,744 


Louisiana 


$5,902,306 


$34,992,340 


Alabama 


$5,769,468 


$36,353,918 


Kentucky 


$5,715,636 


$38,405,563 


Oklahoma 


$4,711,471 


$30,365,438 


North Carolina 


$4,636,868 


$32,967,708 


West Virginia 


$3,545,678 


$25,056,972 


Total (nine states) 


$52,184,783 


$347,466,598 


TOTAL (all states) 


$83,514,292 


$583,057,493 



Exhibit reads: The state of Texas received $7,512,087 in RLIS funding in 2007-08 
and a total of $53,231,150 in RLIS funding from 2002-03 through 2008-09. 



Source: U.S. Department of Education, Office of Elementary and Secondary 
Education, Rural Education Achievement Program (REAP). 



During 2007-08, the RLIS program distributed almost $85 million to 39 states; states distributed RLIS funds to 
1,240 districts (out of the 1,249 districts that were RLIS-eligible), and awards averaged approximately $67,000 per 
district and $33 per pupil. However, until very late in the review process for this report, the RLIS program office 
believed that 1,238 districts received RLIS funds and 1,247 districts were eligible during 2007-08. Consequently, 
the analyses and samples in the report are based on 1,247 RLIS-eligible districts during 2007-08. 
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In 2006-07, the school year on which the RLIS allocations for the 2007-08 school year were 
based, the nine states selected for this study included 670 RLIS-eligible districts, compared with 
617 RLIS districts in the rest of the country. According to program office files, the 670 RLIS- 
eligible districts served 1,598,804 students, about 61 percent of all students in RLIS districts. 
Spending and economically disadvantaged populations for RLIS districts in these nine states 
differ slightly from those of RLIS districts in the rest of the country. In 2006-07, the average 
per-pupil spending was slightly lower in RLIS-eligible districts in these nine states compared 
with RLIS districts in the rest of the country, with averages of $9,350 and $10,376, respectively. 
RLIS-eligible districts in the nine states served a higher percentage of students who qualified for 
free or reduced-price meals than other RLIS-eligible districts, 64 percent compared with 59 
percent in 2006-07. 

It is important to note that these nine states were selected to provide qualitative information 
about state priorities, program administration, and technical assistance, and are not intended to 
represent state implementation across the 39 states receiving funding in the 2006-07 school year. 

Interviews were also conducted with district RLIS staff members in a randomly selected sample 
of five districts in each of the nine sampled states that received RLIS funding in 2007-08. Four 
interviews were completed in Georgia and North Carolina and five interviews were completed in 
the other seven states. As with the interviews with state RLIS staff members, it is important to 
note that these districts were selected to provide qualitative information about RLIS program 
administration and funding priorities and are not intended to represent implementation of the 
RLIS program across all districts receiving funding. 

A content analysis of interview transcripts, extant documents, and other background information 
was performed. A coding framework was developed to review all responses together and 
evaluate each response for possible relationships and for any significant variations. A 
classification matrix was used to organize extant state and district documents into categories of 
analysis, which grouped documents within a relational framework. 



State and District Surveys 

In order to obtain a more representative picture of RLIS priorities and expenditures than could be 
obtained through the state and district interviews and document review, online surveys were 
conducted with state RLIS staff members in all 39 states that received RLIS funds in the 
2007-08 school year and with a randomly selected sample drawn from the 1,247 districts that 
were eligible to receive RLIS funds in the 2007-08 school year. Hard copies of the online 
surveys to which state and district RLIS staff members responded are in Appendices E and F. 

The 39 states that were included in the sample were: Alabama, Arkansas, Arizona, California, 
Colorado, Delaware, Florida, Georgia, Idaho, Illinois, Indiana, Kansas, Kentucky, Fouisiana, 
Massachusetts, Maine, Michigan, Minnesota, Missouri, Mississippi, Montana, North Carolina, 
North Dakota, Nebraska, New Hampshire, New Mexico, New York, Ohio, Oklahoma, Oregon, 
Pennsylvania, South Carolina, South Dakota, Tennessee, Texas, Virginia, Washington, 
Wisconsin, and West Virginia. Survey responses were received from 37 of the 39 states, a 95 
percent response rate; survey responses were not received from Arkansas and Florida. For the 
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district survey, 689 potential district respondents were randomly selected from among the 1,247 
districts that were eligible to receive RLIS funds in the 2007-08 school year. An 84 percent 
response rate was achieved, for a final sample of 580 district RLIS coordinators. 

District-level Extant Data 

Analysis of the extant district data for this final report began by descriptively assessing what 
RLIS districts have looked like over time and by comparing RLIS districts to districts nationally 
and other rural districts in the country. RLIS program data files available on the Department’s 
Web site were used to identify which districts were eligible for RLIS funds. In 28 of the 43 states 
that had ever received RLIS funding, all RLIS-eligible districts received funding in all six years 
of the program. The Common Core of Data (CCD), a federal program that annually collects data 
about all public schools, public school districts and state education agencies in the United States, 
provided district demographic information. The demographic data from the CCD were used to 
compare RLIS districts descriptively to non-RLIS rural districts and all districts nationally. 

The evaluation also sought to model gains in student achievement in RLIS and non-RLIS 
districts by conducting a longitudinal analysis. The analysis used data from the EDEN-EDFacts 
data system, which contains student achievement and accountability data mandated in NCLB. It 
therefore includes data on student achievement on state assessments and data on AYP for all 
districts in the U.S. Student achievement is defined as the percent of students in a district at or 
above proficiency levels established by the state in reading and mathematics as measured by 
state assessments. States determine the percent of students in a district that must be at or above 
proficiency for the district to make AYP each year, with the NCLB goal of all students proficient 
by 2013-14. In 2009, the EDEN-EDFacts data system included achievement data from the 
2005-06, 2006-07, and 2007-08 school years. 

To supplement the achievement data found in the EDEN-EDFacts system, the evaluation drew 
on data collected in the National Eongitudinal School-Eevel Assessment Score Database 
(NESESASD). Funded by the U.S. Department of Education, the NESESASD contains 
assessment scores from roughly 90,000 public schools. The evaluation drew on this data system 
to gather district achievement data from the 2002-03 through 2004-05 school years. The CCD 
data were also used as covariates in modeling the association of the REIS program on student 
achievement. 

In both the EDEN-EDFacts and NESESASD data, the grades assessed and the assessments used 
vary by state. The analysis used data from all grades in which there was assessment data for all 
students in reading or mathematics. To account for the fact that the assessment are not 
comparable from state to state and the grades varied by state, proficiency levels were converted 
to z-scores to assess the relative ranking of REIS districts within their state. 

In addition, both data sources report percent proficient to measure achievement rather than scale 
scores on an assessment. As a result, the analyses only provide information about achievement 
around the proficiency cut point, but do not provide information about changes in performance 
among high- or low-performing students. 
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Chapter 2: State and District Implementation of the RLIS Program 



This chapter contains findings on state and district implementation of the RLIS program. The 
findings are based on the following sources of data: 

• Interviews with RLIS state coordinators in the nine states that received the largest RLIS 
allocations in 2007-08 

• Interviews with RLIS coordinators in 43 RLIS-funded districts in the nine states that 
received the largest RLIS allocations in 2007-08 

• Review of RLIS-related documents obtained from the interview states and districts, such 
as grant application and guidance materials, program data reports, evaluation reports, and 
technical assistance materials 

• A survey of the RLIS state coordinators in the 39 states that received RLIS funding in 
2007-08 

• A survey of RLIS district coordinators in a random sample of the 1,249 districts that 
received RLIS funding in 2007-08 

This chapter incorporates selected findings from the interim report on state implementation of 
the RLIS program. The state implementation findings in the interim report were based on the 
interviews with RLIS coordinators in the nine states that received the largest RLIS allocations in 
2007-08 and on reviews of RLIS-related documents obtained from these states. The results of 
the state coordinator interviews, which were conducted by telephone during the spring of the 

2007- 08 school year, were used to develop the interview protocol for the district coordinator 
interviews and the survey instruments for both the state and district surveys. 

The district interviews were conducted by telephone during the late spring-early summer of the 

2008- 09 school year; RLIS-related documents were collected from the districts during that same 
period. Both the state coordinator survey and the district coordinator survey were conducted 
during the spring of the 2008-09 school year. The guides used in conducting the state and district 
interviews, as well as hard copies of the online surveys to which state and district RLIS staff 
members responded, are in Appendices C-F. 

Key Findings 

Key findings on state and district implementation of the RLIS program include the following: 

• Almost all state and district respondents in the surveys and interviews indicated that they 
set goals for RLIS funds in addition to making AYP, based on state and local needs. 

• When state and district RLIS coordinators were asked to rate the extent to which 
specified goals were a focus of their RLIS programs — in addition to the primary goal of 
making AYP — the most common goals reported for district RLIS programs were 
improving the quality of instruction, improving student achievement in specific subject 
areas, and addressing issues specific to rural location, such as retaining teachers or 
providing distance learning opportunities. 
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• Both district and state survey respondents reported that RLIS funds were primarily used 
to purchase educational technology, support teacher professional development, and, in 
general, support activities authorized under Title I, Part A. Similarly, interviews with 
district RLIS coordinators and reviews of district documents indicated that districts 
primarily used RLIS funds for technology hardware and software, teacher pay, and 
professional development. 

• All nine interviewed state coordinators, and almost all of the surveyed state coordinators 
(37 of 39), reported that all eligible districts in their state received RLIS funds. 

• In telephone interviews, both state and district RLIS coordinators reported that RLIS 
goals and priorities are not separate or distinct from the larger goals and priorities of the 
state or district goals or for those of other federal programs. State and district RLIS 
coordinators reported that their states and districts viewed the RLIS program less as a 
separate program and more as a supplemental funding source to help rural schools meet 
the AYP targets. 

• When asked in the district interview about additional comments on the RLIS program, a 
common response among district coordinators was that the flexibility of the RLIS 
program allowed them to use these funds to meet specific needs in their districts. 

• All nine states in the state interview sample required RLIS districts to engage in a 
comprehensive planning process to identify local needs, and the majority of district 
coordinators in the district interview sample reported undertaking such a process to set 
the goals and priorities for their districts. However, interview and document data showed 
considerable variation among states and districts with regard to the planning tools they 
used and the extent to which districts were directed through the planning process. 

• A majority of district coordinators interviewed and surveyed reported receiving regular, 
knowledgeable, and helpful assistance from state RLIS coordinators in developing their 
applications for RLIS funds. 

State and District Implementation of the RLIS Program 

Following are the findings from the analyses of state and district implementation of the RLIS 
program. Included in the discussion of these findings are brief vignettes that provide detailed 
contextual information obtained from the interviews with district RLIS staff members. 

Goals and Priorities 

In addition to the primary goal of helping rural school districts make AYP, most RLIS- 
funded states and districts set additional goals, based on state and local needs, for their 
RLIS programs. 

Almost all state and district respondents in the surveys and interviews indicated that they set 
goals for RLIS funds in addition to making AYP, based on state and local needs. All nine states 
in the state interview sample required RLIS districts to engage in a comprehensive planning 
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process to identify local needs, and the majority of districts in the district interview sample 
reported undertaking such a process to set the goals and priorities for their district. It is important 
to note that even in states and districts that reported setting additional goals for the RLIS 
program, making AYP took priority over any other goals, and districts with schools in school 
improvement status were required to direct their RLIS funds to helping that school make AYP.^ 

According to state and district survey respondents, interviews with district coordinators, 
and review of district documents, improving the quality of instruction and increasing 
student achievement in particular subject areas were the most commonly reported goals 
for the RLIS program in addition to making AYP. 

While recognizing that making AYP is the primary goal for the use of all RLIS funds, state 
survey respondents were asked to rate the extent to which several specified goals were also a 
focus for RLIS funds in their state (see Exhibit 2). State survey respondents were most likely to 
report that increasing student achievement in a particular subject area and improving the quality 
of instruction in general were a major or moderate focus for RLIS funds. Roughly 97 percent of 
state respondents identified each of these goals as a major or moderate focus in their state. 
Addressing issues specific to rural location, such as retaining teachers or providing distance 
learning opportunities, was a major or moderate focus for 73 percent of the state respondents, 
while reducing the high school dropout rate a major or moderate focus for 50 percent. 



^ Section 1 1 1 1 (b)(2) of ESEA. 
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Exhibit 2 

State-reported Goais by Focus for RLIS Funds in Addition to Making AYP 





Major 

Focus 


Moderate 

Focus 


Minimal 

Focus 


Not a Focus 


Increasing student 
achievement in a particular 
subject area (n = 37) 


73% 


24% 


0% 


3% 


Improving the quality of 
instruction (n = 36) 


53% 


44% 


0% 


3% 


Addressing issues specific to 
rural location, e.g., retaining 
teachers, providing distance 
learning opportunities, etc. 

(n = 36) 


42% 


31% 


17% 


11% 


Reducing the high school 
dropout rate (n = 36) 


25% 


25% 


39% 


11% 


Ensuring that all students will 
be educated in learning 
environments that are safe, 
drag free, and conducive to 
learning (n = 35) 


20% 


31% 


31% 


17% 


Improving the ability of 
English language learners to 
achieve proficiency in English 
and reach high academic 
standards (n = 36) 


19% 


25% 


44% 


11% 



Exhibit reads: Seventy-three percent of state survey respondents reported that increasing 
student achievement in a particular subject area was a major focus in their use of RLIS funds. 

Note: Items are listed in order of “Major Focus” and “Moderate Focus” comhined. 

Source: RLIS State Survey. 



District survey respondents were also asked to rate the extent to which specified goals were an 
additional focus of the use of RLIS funds in their district (see Exhibit 3). Their responses were 
similar to those of the state survey respondents. Improving the quality of instruction in general 
and increasing student achievement in a particular subject area were the most commonly cited 
goals for RLIS funds in addition to making AYP, with roughly 94 and 93 percent of respondents 
respectively identifying these goals as a major or moderate focus in their district. Addressing 
issues specific to rural location, such as retaining teachers or providing distance learning 
opportunities, was a major or moderate focus for 67 percent of the district respondents. Reducing 
the high school dropout rate was more important to district survey respondents than to state 
survey respondents, with 71 percent of the district respondents identifying this goal as a major or 
moderate focus for RLIS funds, compared with 50 percent of the state respondents. 
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Exhibit 3 

District-reported Goais by Focus for RLIS Funds in Addition to Making AYP 





Major 

Focus 


Moderate 

Focus 


Minimal 

Focus 


Not a Focus 


Improving the quality of 
instruction (n = 564) 


73% 


21% 


3% 


2% 


Increasing student achievement in 
a particular subject area (n = 559) 


70% 


23% 


4% 


3% 


Reducing the high school dropout 
rate (n = 561) 


38% 


33% 


16% 


13% 


Addressing issues specific to rural 
location, e.g., retaining teachers, 
providing distance learning 
opportunities, etc. (n = 557) 


36% 


31% 


15% 


17% 


Ensuring that all students will be 
educated in learning environments 
that are safe, drug free, and 
conducive to learning (n = 553) 


35% 


30% 


19% 


16% 


Improving the ability of English 
language learners to achieve 
proficiency in English and reach 
high academic standards (n = 552) 


22% 


24% 


22% 


32% 



Exhibit reads: Seventy percent of district survey respondents reported that increasing student 
achievement in a particular area was a major focus of their district’s use of RLIS funds. 

Note: Items are listed in order of “Major Focus” and “Moderate Focus” comhined. 

Source: RLIS District Survey. 



In the interviews with district RLIS coordinators, additional information emerged about goals 
and priorities for RLIS funds. Seventeen of the 43 district coordinators interviewed reported that 
a priority for their district was increasing student achievement in a particular subject area, with 
eight reporting that they focused on increasing student achievement in mathematics and reading 
and four reporting that they focused on increasing achievement in mathematics and science. Four 
of the 43 district coordinators interviewed reported focusing all of their RLIS funds on 
improving achievement in one school. Thirteen of the 43 districts reported focusing on 
improving student achievement in a targeted school level, with 1 1 of the 13 focusing on middle 
or high schools. Three districts reported improving high school graduation rates as a priority for 
RLIS funds. One district coordinator who reported focusing RLIS funds at the middle or high 
school level explained that their middle and high schools were not Title I schools and therefore 
did not receive as much federal funding as the elementary schools in the district. The middle and 
high schools had identified initiatives to improve student achievement but, prior to receiving 
RLIS funds, could not afford to implement those programs. The RLIS funds were used to fill this 
gap and support those initiatives. 
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Uses ofRLIS Funds 

Districts integrated their RLIS funds into ongoing school improvement activities to help 
schools meet AYP targets. 

A common response from the district coordinators interviewed was that RLIS funds were used to 
provide supplemental programs and services that were not covered under other federal programs 
to support school improvement efforts already in place. Districts reported helping schools to 
meet AYP targets by integrating RLIS funds into ongoing school improvement activities. 

The following example demonstrates how one district used RLIS funds to supplement its efforts 
to improve the quality of instruction districtwide. 



Rockcastle County, Ky. 

One of the goals of the Rockcastle County Schools was to improve technology and to 
provide training on the use of technology in instruction. With the implementation of 
the Kentucky Education Reform Act in 1990, money for technology flowed into rural 
districts. The district supervisor of instruction reported that this money made the 
district feel wealthier than it had ever been, but soon the district learned that 
technology was expensive and quickly became outdated. Yet this increase in 
technology changed the way the district approached school improvement. District 
leaders shifted their thinking from focusing in broad terms about improvements in 
reading and math to asking how the district could better use technology to improve its 
efforts in reading and math. This required training teachers in using technology for 
instruction. As a rural district that was far from the professional development 
opportunities available in more urban areas, the district examined ways to bring 
professional development for teachers to the county. With support from multiple 
grants, the district remodeled an old bus garage to create a training center. RLIS funds 
provided money for computers for the center. The district reported that the center was 
booked almost every day for training activities. The center provided an economical 
approach to teacher training. It was located near the district office and therefore could 
be staffed by district personnel. The district saved on the expense of time and travel 
for teachers by utilizing online training and district and school personnel to deliver 
training. 






Data from state and district surveys, district interviews and document review show that 
districts used their RLIS funds primarily to purchase educational technology, to support 
professional development and to support activities authorized under Title I, Part A. 



State survey respondents were asked to rate the extent to which certain activities were a focus of 
the use of RLIS funds in their state (see Exhibit 4). Respondents were most likely to identify 
purchasing educational technology (92 percent), supporting teacher professional development 
(90 percent), and supporting activities authorized under Title I, Part A (88 percent), as a major or 
moderate focus of their RLIS funds. 



Exhibit 4 

District Activities Using RLIS Funds by Focus, Reported by States 





Major 

Focus 


Moderate 

Focus 


Minimal 

Focus 


Not a Focus 


Educational technology, 
including software and 
hardware (n = 37) 


51% 


41% 


8% 


0% 


Teacher professional 
development (n = 37) 


49% 


41% 


11% 


0% 


Activities authorized under 
Title I, Part A (n = 35) 


54% 


34% 


6% 


6% 


Parental involvement 
activities (n = 36) 


17% 


42% 


25% 


17% 


Teacher recmitment and 
retention, including the use of 
signing bonuses and other 
financial incentives (n = 37) 


22% 


32% 


24% 


22% 


Activities authorized under 
the Safe and Dmg-Free 
School Program (n = 37) 


11% 


38% 


35% 


16% 


Language instruction for 
ELL/LEP students (n = 37) 


11% 


24% 


51% 


13% 



Exhibit reads: Fifty-one percent of state survey respondents reported that educational 
technology, including software and hardware, was a major focus of RLIS funds in their state. 

Note: Items are listed in order of “Major Focus” and “Moderate Focus” combined. 

Source: RLIS State Survey. 







District survey respondents were also asked to rate the extent to which various activities were a 
focus in their use of RLIS funds (see Exhibit 5). Respondents were most likely to report using 
REIS funds to support activities authorized under Title I, Part A (76 percent), to purchase 
edueational technology, including hardware and software (75 percent), and to support teacher 
professional development (68 percent). Respondents were least likely to report using REIS funds 
to support teacher recruitment and retention, with 62 percent of respondents reporting that this 
was not a focus on their REIS spending. 



Exhibit 5 

District Activities Using RLIS Funds by Focus, Reported by Districts 





Major 

Focus 


Moderate 

Focus 


Minimal 

Focus 


Not a Focus 


Activities authorized under 
Title I, Part A (n = 559) 


52% 


24% 


9% 


15% 


Educational technology, 
including software and 
hardware (n = 565) 


52% 


23% 


12% 


14% 


Teacher professional 
development (n = 564) 


37% 


31% 


15% 


18% 


Parental involvement 
activities (n = 556) 


14% 


25% 


30% 


31% 


Activities authorized under 
the Safe and Dmg-Free 
School Program (n = 553) 


15% 


21% 


24% 


40% 


Language instruction for 
ELL/LEP students (n = 552) 


12% 


20% 


22% 


46% 


Teacher recmitment and 
retention, including the use of 
signing bonuses and other 
financial incentives (n = 556) 


9% 


10% 


19% 


62% 



Exhibit reads: Fifty-two percent of district survey respondents reported that activities 
authorized under Title I, Part A, and educational technology were a major focus of their 
use of RLIS funds. 

Note: Items are listed in order of “Major Focus” and “Moderate Focus” combined. 
Source: RLIS District Survey. 
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District interview responses — from the non-randomly-selected sample of 43 districts in the nine 
sample states — were largely consistent with the state and district survey responses. The district 
RLIS coordinators interviewed reported that they used RLIS funds primarily for technology 
hardware and software, teacher pay, and professional development (see Exhibit 6). 



Exhibit 6 

District-reported Use of RLIS Funds 



Use of RLIS Funds 


Number of 
Districts 
(n=43) 


Educational technology 


28 


Teacher pay 


24 


Professional development 


21 


Materials 


15 


Transportation for after-school 
program 


2 



Exhibit reads: Teacher pay is a use of RLIS funds for 
24 districts interviewed. 



Source: RLIS district coordinator interviews. 



The most commonly reported use of district RLIS funds by the 43 district coordinators 
interviewed was to purchase educational technology: 28 districts reported using funds for 
hardware, software, or both. Of these 28 districts, 13 reported purchasing computers for 
classrooms or computer labs, and eight reported purchasing Smart Boards. Other technology 
equipment purchases included projectors and calculators. Nineteen districts reported using RLIS 
funds to purchase computer software aimed at improving student achievement. Districts reported 
using the funds to renew licenses for programs and to purchase new programs. Examples of the 
types of programs that were purchased included predictive assessments to help teachers monitor 
the progress of students, subject matter software and programs that included the ability for 
parents to access information on the academic progress of their children. 

Eighteen of the 43 district RLIS coordinators interviewed reported that improving technology 
was key to helping them to achieve the broader goal of making AYP. District coordinators that 
reported using RLIS funds to acquire or update technology described technology as a critical 
component in their school improvement efforts. 
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The following example illustrates how one district used computer software to help it meet its 
goal of improving student achievement in reading and math. 



Perry County, Miss. 

During the first couple of years of RLIS funding, Perry County targeted low- 
performing schools and districts. During the 2007-08 school year, the district decided 
to invest its RLIS funds in computer software that would benefit all schools and all 
students in the district. The district purchased a reading intervention program designed 
to develop and strengthen students’ memory, attention, processing rate, and 
sequencing. The program was designed to support existing curriculum, not replace it. 
The district also purchased a math curriculum software program. Computer labs were 
then placed in each school and staffed with an instructor and an assistant. Students 
were scheduled in the lab every day and encouraged to work with these programs at 
their own pace. The programs monitored the students’ progress and helped teachers 
target instruction to student needs. 



The second most commonly reported category of spending by district coordinators interviewed 
was pay for personnel who provided supplemental services. Examples include pay for tutors, 
technology coordinators, parent involvement coordinators, intervention specialists, and 
counselors. 
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RLIS Administrative Processes 



The vast majority of state survey respondents reported that they use RLIS funds for 
administrative purposes. 

Almost all states (35 of 37) reported that they used some RLIS funds for administrative purposes. 
Of the 35 states that reported using some funds for administrative purposes, 28 reported that they 
used 5 percent of state RLIS funds for this, six reported using less than 5 percent, and one did not 
specify the amount set aside for this use. 

The vast majority of state survey respondents and all nine interviewed state coordinators 
reported that all eligible districts in their state receive RLIS funds. 

By statute, states may award subgrants to eligible districts either by formula or competitively. 

The nine states included in the interview sample allocated RLIS funds by formula to all eligible 
districts. 

Almost all state coordinators (37 of 39) reported that all eligible districts in their state receive 
RLIS funds. Roughly half of state survey respondents reported that eligible districts must 
complete a separate RLIS application (see Exhibit 7). Another 41 percent of state survey 
respondents reported that eligible districts must complete an RLIS application that is a part of a 
comprehensive district improvement plan or other district application. Only 5 percent of state 
survey respondents reported that there is no formal process or protocol eligible districts must 
follow to receive state funds. 



Exhibit 7 

Protocois Eiigibie Districts Must Foiiow, by Percent, to Receive RLIS Funds 





Percent of States 
(n=37) 


A separate RLIS-specific application form or process 


51% 


An application for RLIS funds that is a part of a comprehensive district 
improvement plan or other district application for funds 


41% 


No formal process or protocol; all districts receive all funds for which they 
are eligible 


5% 


Other 


3% 



Exhibit reads: Fifty-one percent of state survey respondents reported that RLIS-eligible 
districts complete separate RLIS-specific application form or process in order to receive 
the funds for which they are eligible. 

Source: RLIS State Survey. 
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State survey respondents were asked how they inform districts that they are eligible to receive 
funds through the RLIS program. E-mail was the most common form of communication cited 
(62 percent), followed by posting the information on a Web site (54 percent) and sending eligible 
districts a letter (51 percent). 

District RLIS coordinators interviewed reported a variety of district offices assigned to 
implement the RLIS program. 

Of the 43 district coordinators interviewed, 14 reported that their district RLIS program was 
implemented by a federal programs officer, whose sole responsibility was overseeing the 
administration of federal programs. Two of the districts reported that their program was 
administered by a Title I coordinator with similar responsibilities as a federal programs officer. 
The other 27 districts reported that the RLIS program was administered by district personnel 
whose responsibilities included more than implementation of federal programs. The titles of 
those responsible for RLIS administration included superintendent, assistant superintendent, 
principal, administration assistant, director of districtwide services, director of education support 
and programs, supervisor of instruction, and director of technology. 

Typical of small districts, the majority of administrators interviewed reported that they wear 
many hats and have a wide range of responsibilities. As illustrated by the example below, job 
descriptions provided by district RLIS coordinators often included responsibilities that would be 
covered by three or four positions in a larger district. 



Bienville Parish, La. 

Tony Hough is the director of instructional support for the Bienville Parish Schools, 
a school district of approximately 7,400 students. He oversees the district’s 
technology, data management, testing, and accountability programs. As technology 
coordinator. Hough is responsible for the upkeep of all technology in the district, 
identifying new technology, and dealing with E-rate. As data management 
coordinator, he oversees the system’s electronic gradebook and transcript system 
and ensures that teachers are trained in these systems. As accountability and testing 
coordinator, he is responsible for all state-mandated testing in the district. Without 
an office of federal programs, when the RLIS program was introduced, the district 
assigned responsibility for that program to Hough because of the district focus on 
technology and his role as technology coordinator. 



While district coordinators described varying levels of involvement in the process of developing 
priorities and strategies for the RLIS funds, their primary role was to ensure that the district’s 
RLIS funds were administered properly. All 43 district coordinators interviewed reported 
working closely with senior administrators and advisory groups in their districts in order to align 
the district’s RLIS spending with its identified goals and priorities, and ensure that the funds 
were spent in accordance with federal regulations. 
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Districts reported that the flexibility of the RLIS program allowed them to use the funds 
effectively to meet the needs of the district. 

When asked if they had any additional comments on the RLIS program, the most common 
response from district coordinators included praise for the flexibility of the program. District 
coordinators explained that the structure of the program allowed them to easily target the RLIS 
funds toward the district’ s top needs in their efforts to improve student achievement. 

State and district coordinators reported that RLIS was the last of the federal grants to he 
allocated, which, for a district close to the poverty threshold for RLIS funding, may affect 
that district’s ability to plan its budget. 

Although district budget planning is usually done in the early summer, the notification from ED 
to the state and thus, from the state to the district, regarding the final RLIS allocations often does 
not come until after school has started in the fall. When asked if they had any suggestions for 
improving the RLIS program, district coordinators explained that earlier notification of the RLIS 
allocations could improve their budget planning. State coordinators also expressed concern about 
how the timing of the RLIS notification might affect “borderline” districts that may be above the 
poverty threshold one year but fall below it the next, as such districts would not know their RLIS 
funding status at the time they put together their annual budgets. 

In addition to monitoring the implementation of the RLIS program by the districts in their 
states, about one-third of state survey respondents reported that they had conducted 
evaluations of the RLIS program. 

Twenty-one of the 43 interviewed districts reported having received a monitoring visit from the 
state at some point; the time frame for these visits was not specified. District coordinators 
described the visits as “comprehensive,” covering all federal programs. In addition, 12 of the 37 
state survey respondents reported that they had evaluated the RLIS program in their state. Of the 
12 states that reported having conducted an evaluation, four reported that a report or other 
document was produced as a part of the evaluation. None of these reports were available online. 



District Planning and Needs Assessment 

All nine interviewed states required RLIS districts to engage in a comprehensive planning 
process to identify local needs, and the majority of interviewed districts reported 
undertaking such a process to set the goals and priorities for their district. 

State coordinators in all nine of the interviewed states reported that that they viewed the RLIS 
program as one of several federal funding sources available to help rural, low-income districts 
reach the goals of improving student achievement and making AYP, and that they encouraged 
districts to use all available federal funds to coordinate local efforts to achieve these goals. State 
and district RLIS coordinators did not view the RLIS program as a separate program. In 
telephone interviews, both state and district RLIS coordinators reported that RLIS goals and 
priorities are not separate or distinct from the larger goals and priorities of the state or district 
goals or for those of other federal programs. 
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All of the interviewed state coordinators reported that their state had a comprehensive planning 
process that was designed to help districts identify areas of local need and select appropriate 
strategies across funding sources to address those needs. As reported by the state coordinators 
and verified by state documents, there was considerable variability among the nine states with 
regard to the planning tools they used and to what extent they directed districts through the 
planning process. The states’ comprehensive planning processes typically included a 
consolidated application that described how federal funds, including RLIS funds, would be used 
together to implement and evaluate those strategies. Seven of the nine states used a consolidated 
application for all federal funds, and four of those used an online application. In the two states 
that did not use a consolidated application, districts completed individual applications for each 
federal program, including RLIS. Six of the nine state coordinators interviewed stated that their 
state developed a needs assessment process or tool to help districts identify their primary needs 
in relation to improving student achievement. Twenty-five of the 3 1 districts that reported 
undertaking a comprehensive needs assessment were in the six states that had developed a 
process or tool to assist districts in conducting needs assessment. 

District interview and document data showed considerable variation among districts with regard 
to the planning tools they used, the structure of their needs assessment, and the types of data 
collected. District needs assessments ranged from formal ones — following required guidelines 
from the state as to the kinds of data collected and the instruments used — to informal 
conversations with principals about their needs. 

District processes for identifying goals and priorities for using RLIS funds reported by 
district coordinators fell into two categories: Decisions made primarily by the RLIS 
coordinator or decisions made by a small district advisory team. 

The majority of district RLIS coordinators interviewed (31 of 43) reported that their district 
undertook some kind of comprehensive needs assessment annually and that decisions about the 
overall goals and priorities of the district — not just decisions related to the RLIS program — were 
based on that needs assessment. District RLIS coordinators reported varying types of structures 
for their needs assessments. The most structured processes included surveys of district and 
school personnel and parents and students. These assessments also included regularly scheduled 
meetings to analyze and discuss district data. The least structured processes included district 
coordinators getting input, in a more informal way, from principals about their needs. 
Involvement in the process also varied from a small group of district administrators (sometimes 
only one or two) to a more inclusive process that involved district administrators, school 
personnel, students, parents, and community members. The districts also reported differences in 
how they used the information from the needs assessment to determine the district’s RLIS goals 
and priorities. Districts in which the RLIS coordinator was the superintendent or a school 
principal were more likely to make decisions about goals and priorities for the programs 
themselves or in consultation with one other administrator (nine districts). Other districts relied 
primarily on a district advisory team to review the information derived from the districtwide 
needs assessment to identify goals and priorities (five districts). These district advisory teams 
included such personnel as superintendent, assistant superintendents, chief financial officer, 
principals, teachers, and curriculum specialists or coaches. 
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The following example illustrates how one district conducted a needs assessment based on 
requirements and guidelines from the state. 



Harrison County, W.Va. 

School districts in West Virginia are required to develop a five-year strategic plan. Included in 
the plan are the mission and goals of the school system to improve school system performance 
and progress. The plans are revised annually and must include performance measures and 
action to be taken to meet each measure.* 

Harrison County, like other districts in West Virginia, relies on its five-year strategic plan to 
determine how funds from various federal programs will be allocated. By state policy, tbe 
goals of tbe plan must focus on improved student acbievement. The district’s broad data 
gathering process included data from an array of sources including state tests, end-of-course 
exams, and ACT and SAT results. Additionally, the district analyzed the overall culture, 
conditions, and practices of the school system through examination of monitoring reports, 
questionnaires and observations completed by staff or external evaluators. The district also 
included an analysis of external trend data (demographic data) in its needs assessment. 
Harrison County utilized a Curriculum Team and Strategic Planning Committee that included 
the superintendent, assistant superintendents, administration assistants, and math and literacy 
coaches to review the data and prioritize strategic issues to develop the five-year strategic 
plan. Planning for the use of RLIS funds was part of this comprehensive process that drew 
from a variety of funding sources to support the strategies aimed at meeting district goals for 
student achievement. 

* Source: A Process for Improving Education: Performance Based Accreditation System, Electronic County and 
School Strategic Improvement Plan Process §126-13-9. 2007. 



State Guidance and Assistance 

The interviewed district coordinators reported receiving knowledgeable and helpful 
assistance from state RLIS coordinators in developing their applications for RLIS funds. 

Interviewed district coordinators described how state coordinators worked closely with them as 
needed to develop a proposal that was in line with federal regulations. The majority of districts 
(30) reported that assistance was provided primarily on an as-needed basis through phone calls 
and e-mails. Districts also reported receiving guidance and assistance through statewide 
conferences (18) and regional meetings (5). Thirteen districts reported receiving documents 
regarding the RLIS program guidelines, and five reported obtaining information about the RLIS 
program through the state Web site. 

District reports confirmed what state coordinators had previously reported in interviews about 
their provision of guidance and assistance to districts. State coordinators had reported that their 
directions to districts about RLIS funds ranged from simply restating the federal statute to 
providing assistance in completing RLIS funding applications and encouraging districts to use 
RLIS funds to support specific activities, such as professional development or leadership 
development. Similarly, district coordinators reported that the state coordinators had provided 
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technical assistance in completing RLIS applications, assisting with goal-setting, identifying 
effective strategies for using RLIS funds, and assisting with program evaluation (see Exhibit 8). 



Exhibit 8 

State Assistance Provided to RLIS Districts 



Assistance From State 


Number of 
Districts 

(n=43) 


Application 


34 


Goal setting 


27 


Program evaluation 


12 


Identifying effective strategies 


11 



Exhibit reads: Thirty-four of the 43 districts interviewed reported 
receiving state assistance with regard to the RLIS application process. 

Source: RLIS district coordinator interviews. 



All 37 of the state survey respondents reported that they communicated with districts in their 
state specifically about the RLIS program. State administrators were asked about topics with 
which they communicated with districts (see Exhibit 9). A vast majority (97 percent) reported 
communicating with districts about eligibility for REIS funds, the REIS application process, and 
allowable costs. Roughly 68 percent reported providing districts with assistance in developing 
REIS activities. 



Exhibit 9 

Topics of RLIS-reiated Communications Between States and Districts 





Number of 
States 
(n = 37) 


Eligibility for RLIS funds 


36 


RLIS application process 


36 


Allowable costs 


36 


Assistance in developing RLIS 
activities 


25 


Other 


3 



Exhibit reads: Thirty-six of 37 state survey respondents reported communicating with 
districts about eligibility for RLIS funds. 

Source: RLIS State Survey. 
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There was some variation in the frequency with which state survey respondents reported 
communicating with districts about the RLIS program. Most districts reported that they 
communicated with districts one or two times a year (43 percent) or monthly or every other 
month (41 percent). Sixteen percent of districts reported communicating with districts about the 
RLIS program more often than once a month. 

All 37 states responding reported that they provide technical assistance to districts that receive 
RLIS funds. State survey respondents were most likely to report that information about the RLIS 
program was available on their Web site (72 percent) and that they provided a district budget 
review of RLIS fund appropriations (69 percent) (see Exhibit 10). 



Exhibit 10 

Types of Technicai Assistance That 
States Provide Districts Receiving RLIS Funds 





Number of States 
(n = 36) 


Information provided on RLIS program on state Web site 


26 


District budget review of RLIS fund appropriations 


25 


Checklist on appropriate use of funds 


18 


Conference or workshop presentations on RLIS program 


15 


Handbook or guidelines on the appropriate use of RLIS funds 


14 


Workshops or conference sessions devoted to the receipt of RLIS funds 


10 


Other 


II 



Exhibit reads: Twenty-six of the 36 state survey respondents reported that they provided 
information on their RLIS program on a state Web site. 

Source: RLIS State Survey. 



Perceived Helpfulness of State Technical Assistance 

District coordinators interviewed and surveyed reported receiving regular, knowledgeable, 
and helpful assistance from state RLIS coordinators in developing their RLIS applications. 

In the district survey, approximately 81 percent of survey respondents reported that they 
communicate with their state education agency specifically about the RLIS program. Among 
these districts, survey respondents were most likely to report communicating with their state 
education agencies about allowable costs under the RLIS program (86 percent) (see Exhibit II). 
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Exhibit 11 

Topics About Which Districts Reported 
Communicating with State Education Agencies, by Percent 





Percent of Districts 
(n = 469) 


Allowable costs 


86% 


RLIS application process 


75% 


Planning/developing RLIS-funded activities 


70% 


Eligibility for RLIS funds 


67% 



Exhibit reads: Eighty-six percent of districts survey respondents reported communicating 
with their state education agency about eligibility for REIS funds. 

Source: REIS District Survey. 

Among district survey respondents who reported communicating with their state education 
agencies about the RLIS program, 53 percent reported that this communication occurred rarely, 
one or two times per year. Five percent reported that this communication occurred more than 
once every month, with the remainder reporting occasional contact. District survey respondents 
were also asked to rate the extent to which they have received technical assistance from their 
state education agency for the RLIS program. Most survey respondents reported that they 
received a moderate (38 percent) or minimal (36 percent) amount of technical assistance from 
their state education agency. About 8 percent reported not receiving any technical assistance. 
District survey respondents that reported that they at least received a minimal amount of 
technical assistance from the state were asked how this information was used. Respondents were 
most likely to report that they used assistance or information from the state to learn about or 
check on the appropriate use of RLIS funds (85 percent) or to complete their RLIS application or 
comprehensive district plan to receive RLIS funds (83 percent) (see Exhibit 12). 



Exhibit 12 

District Use of Information or Technicai Assistance 
Provided by the State Education Agency, by Percent 





Percent of Districts 
(n = 533) 


To learn about/check on appropriate use of funds 


85% 


To complete our application or comprehensive plan for funds 


83% 


To identify areas needing attention 


35% 


To come up with new ideas for spending RLIS funds 


32% 


Other 


2% 



Exhibit reads: Eighty-five percent of district survey respondents reported that they used 
information or technical assistance provided by the state education agency to learn about or 
check on appropriate use of funds. 

Source: REIS District Survey. 
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Finally, district survey respondents who reported receiving some technical assistance from the 
state were asked to rate the helpfulness of various state technical assistance activities. The 
majority of survey respondents received all of the sources of support about which they were 
asked (see Exhibit 13). They were least likely to report that the state provided conferences or 
workshops on the REIS program or workshops or conference sessions devoted to the receipt of 
REIS funds. District survey respondents that received each type of technical assistance generally 
found them helpful, with only a small fraction of respondents reporting that they received 
support that they did not find helpful. Respondents were most likely to report that checklists on 
the appropriate use of REIS funds were helpful. 



Exhibit 13 

Heipfuiness of Various Forms for State Technicai Assistance, by Intensity 





Very 

Helpful 


Moderately 

helpful 


Minimally 

helpful 


Received, 
not helpful 


Did not 
receive 


Handbook or guidelines 
on the appropriate use of 
RLIS funds (n = 526) 


25% 


35% 


16% 


2% 


22% 


Conference or workshop 
presentations on RLIS 
program (n = 519) 


24% 


25% 


12% 


1% 


39% 


Information provided on 
RLIS program on state 
Weh site (n = 520) 


21% 


41% 


20% 


4% 


14% 


Checklist on appropriate 
use of funds (n = 523) 


37% 


36% 


11% 


2% 


14% 


Workshops or conference 
sessions devoted to the 
receipt of RLIS funds 
(n = 5I4) 


22% 


22% 


11% 


4% 


42% 


District budget review of 
RLIS fund appropriations 


29% 


36% 


20% 


3% 


13% 



(n = 511) 



Exhibit reads: Twenty-five percent of district survey respondents reported that they found a 
handbook or guidance on the appropriate use of RLIS funds very helpful. 

Source: RLIS District Survey. 







Perceptions of Progress in Meeting Goals 

State coordinators were asked to rate their progress toward RLIS goals and objectives, including 
the goal of making AYP (see Exhibit 14). The majority (78 percent) of the state survey 
respondents rated their progress toward RLIS goals as moderate. None felt that no progress had 
been made toward RLIS goals. 



Exhibit 14 

State Assessment of Progress Meeting RLIS Goais, 
Inciuding Making AYP, by Number of States 





Number of States 
(n = 37) 


Goals and objectives have 
been accomplished 


4 


Moderate progress 


29 


Minimal progress 


4 


No progress 


0 



Exhibit reads: Four of the 37 state survey respondents reported 
that their RLIS goals and objectives had been accomplished, 
including making AYP. 

Source: RLIS State Survey. 
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State coordinators were then asked to rate the extent to which RLIS funding contributed to their 
state’s progress toward specific RLIS goals and objectives (see Exhibit 15). State survey 
respondents were most likely to report that RLIS funds made a moderate or major contribution in 
making AYP (89 percent), improving the quality of instruction (89 percent), and increasing 
student achievement in a particular subject area (89 percent). 

Exhibit 15 

State Assessment of the Contributions of the RLIS Program 
to Meeting Specific RLIS Goals and Objectives, by Intensity 



Making AYP (n = 37) 

Improving the quality of 

instruction (n = 36) 

Increasing student 
achievement in a particular 

subject (n = 36) 

Addressing issues specific to 
rural location, e.g., retaining 
teachers, providing distance 
learning opportunities, etc. 

(n = 36) 

Ensuring that all students will 
be educated in learning 
environments that are safe, 
drug free, and conducive to 
learning (n = 35) 

Reducing the high school 

dropout rate (n = 36) 

Improving the ability of 
English language learners to 
achieve proficiency in 
English and reach high 
academic standards (n = 35) 

Exhibit reads: Twenty-four percent of state survey respondents reported that RLIS funds made 

a major contribution to their meeting the goal of making AYP. 

Source: RLIS State Survey. 



Major 


Moderate 


Minimal 


No 


Contribution 


Contribution 


Contribution 


Contribution 


24% 


65% 


11% 


0% 


36% 


53% 


11% 


0% 


22% 


67% 


8% 


3% 


17% 


53% 


22% 


8% 


14% 


29% 


51% 


6% 


8% 


31% 


56% 


6% 


9% 


23% 


57% 


11% 
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District survey respondents were also asked to assess their district’s overall progress toward 
RLIS goals, including making AYP (see Exhibit 16). Similar to the state survey respondents, the 
majority of district survey respondents reported that moderate progress has been made toward 
RLIS goals. 



Exhibit 16 

District Assessment of Progress Meeting RLIS Goais, 
Inciuding Making AYP, by Percent of Districts 





Percent of Districts 
(n = 573) 


Goals and objectives have 
been accomplished 


22.0% 


Moderate progress 


76.0% 


Minimal progress 


3.0% 


No progress 


0.2% 



Exhibit reads: Twenty -two percent of district survey respondents reported that their 
RLIS goals and objectives had been accomplished, including making AYP. 

Source: RLIS District Survey. 
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District respondents who reported making at least minimal progress toward RLIS goals were 
asked to rate the contribution the RLIS program made toward helping them to achieve specific 
goals and objectives. The RLIS program was seen as making the greatest contribution to making 
AYP and improving the quality of instruction, with 90 percent of district survey respondents 
reporting that the RLIS program made a major or moderate contribution to meeting each of these 
two goals (see Exhibit 17). 



Exhibit 17 

District Assessment of the Contributions of the RLIS Program 
to Meeting Specific RLIS Goals and Objectives, by Intensity 





Major 

Contribution 


Moderate 

Contribution 


Minimal 

Contribution 


No 

Contribution 


Making AYP (n = 570) 


48% 


42% 


8% 


1% 


Improving the quality of 
instruction (n = 566) 


54% 


36% 


7% 


2% 


Increasing student 
achievement in a particular 
subject (n = 560) 


50% 


39% 


9% 


2% 


Reducing the high school 
dropout rate (n = 561) 


19% 


40% 


28% 


13% 


Addressing issues specific to 
rural location, e.g., retaining 
teachers, providing distance 
learning opportunities, etc. 

(n = 560) 


23% 


33% 


22% 


21% 


Ensuring that all students will 
he educated in learning 
environments that are safe, 
drag free, and conducive to 
learning (n = 557) 


23% 


31% 


26% 


21% 


Improving the ability of 
English language learners to 
achieve proficiency in 
English and reach high 
academic standards (n = 559) 


13% 


25% 


27% 


34% 



Exhibit reads: Forty-eight percent of district survey respondents reported that RLIS funds 
made a major contribution to their meeting the goal of making AYP. 

Source: RLIS District Survey. 



The district coordinators interviewed explained that while they believed their district had made 
progress toward meeting their RLIS goals, it was not possible to attribute that progress 
completely to the RLIS funded activities, as those activities were part of a broader effort aimed 
at improving student achievement and making AYP. 
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Chapter 3: Characteristics of RLIS -eligible Districts 

This chapter presents information on the demographic characteristics of eligible RLIS districts^ 
and compares those characteristics to those of the national average and the average of rural 
districts that did not qualify for RLIS funds. In contrast to Chapter 2, all findings in this chapter 
draw on data from all RLIS districts across the country. The characteristics of students in RLIS 
districts remained relatively stable in the four school years examined, 2003-04 to 2006-07. 

Key Findings 

Key findings on district characteristics include the following: 

• On average, RLIS districts tended to have more students than other rural districts and 
fewer students than all districts nationally. 

• RLIS districts were more concentrated in the South than other rural districts and districts 
nationally. 

• Student-teacher ratios in RLIS districts were slightly lower than in other districts 
nationally but slightly higher than in other rural districts. 

• Total per-pupil spending in RLIS districts slightly increased between 2003-04 and 
2006-07. However, per-pupil spending remained substantially lower in RLIS districts 
than in all districts nationally or in other rural districts, indicating the RLIS program is 
targeting districts as intended. 

• Compared with districts nationally, students in RLIS districts were more likely to be 
white, black, American Indian or Alaskan Native and less likely to be Hispanic, Asian or 
Pacific Islander. Compared with other rural districts, students in RLIS districts were less 
likely to be white and more likely to be black or Hispanic. 

• RLIS districts served a higher proportion of students who qualified for free or reduced- 
price meals and a slightly higher proportion of students who had an Individualized 
Education Program (lEP) compared with districts nationally and non-REIS rural districts. 

• Students in REIS districts were less likely to be limited English proficient than students 
nationally but more likely to be limited English proficient than students in non-REIS 
rural districts. 



® This chapter relies on information provided by the RLIS program office that identifies which districts were eligible 
to receive RLIS funds in each school year. While there may be some districts that were eligible for the program but 
did not receive funds, consultations with the program office as well as independent investigation suggest that, with 
few exceptions, RLIS-eligible districts did, in fact, receive RLIS funds. For this reason, RLIS-eligible is used as a 
proxy for RLIS-funded districts. 
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Data Sources 

Extant data analyzed in this chapter come from CCD and REIS program data. The data files 
covered school years 2003-04 through 2006-07 and included a number of district-level variables 
regarding student population and performance data. Demographic data were analyzed for REIS 
districts for each year of available data. When applicable, demographic data were compared 
among REIS districts, all U.S. school districts, and non-REIS districts with NCES locale codes 
of 6, 7, or 8. While the data files included a number of common variables used annually by 
schools and districts, it was necessary to address several data limitations: 

• Several districts did not report values for some of the variables. When they did not report 
information, these districts were not included in the analysis. Exhibit A-1 in Appendix A 
reports the total number of districts included in the data file for each school year. The 
number of districts that were missing values for the variables used in specific analyses 
can be determined by comparing the N reported for the analysis (Exhibits A-2 through 
A- 17) from the appropriate total number of districts reported in Exhibit A-1. Eor 
example, in 2004-05, 1,123 REIS districts reported data on total district population 
(Exhibit A-2); 1,127 districts are included in the file (Exhibit A-1). Therefore, four REIS 
districts were missing data regarding total district population. In addition, for the data 
concerning economically disadvantaged, limited English proficient, and students with 
lEPs, only those districts which reported both a count for each special population and a 
count for total student population variable were included in the analysis. 

Comparison With Other Districts 

Throughout this section, data on two groups of districts are presented to provide context for 
interpreting the data on REIS districts. Eirst, data are presented on non-REIS rural districts 
(referred to as non-REIS and other rural districts). These rural districts (NCES locale code 6, 7, 
or 8) would not be eligible for the REIS program for one of two reasons: they are eligible for 
SRSA or they are serving a district population with fewer than 20 percent of residents below the 

n 

poverty line. Second, data are presented on all districts nationally. In 2006-07, approximately 7 
percent of districts in the nation qualified for REIS; similar percentages of districts qualified 
throughout the 2003-04, 2004-05, and 2005-06 school years. Most of the data in this section are 
presented graphically. Data tables with additional information on these exhibits, including the 
number of cases included in each exhibit, can be found in Appendix B. 



^ In 2006-07, CCD started using a new locale code designation, and information on which districts were a 6,7, or 8 
under the old classification system was not longer available. To draw a comparison non-RLIS rural group in this 
year, locale code classifications from 2005-06 were used to select the comparison group. 
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Characteristics of Districts 
District Size 

The total student population for all district types remained relatively stable from 2003-04 to 
2006-07. During this period, RLIS districts had an average reported population of about 2,200 
students, smaller than the national average of roughly 3,200 students. However, the average 
population of RLIS districts was larger than that of non-RLIS rural districts, which enrolled 
approximately 1,100 students (see Exhibit 18). This difference between the average size of RLIS 
and non-RLIS rural districts is most likely a result of how the SRSA and RLIS programs 
determine program eligibility. To qualify to receive SRSA funding, a rural district must serve no 
more than 600 students or qualify through alternate population density measures. RLIS program 
eligibility guidelines set no limits on district size but exclude districts that are eligible for SRSA 
funding. Roughly half of the non-RLIS rural districts qualified for SRSA. With at least half of 
the non-RLIS rural districts having fewer than 600 students, it stands to reason that the average 
district size would be lower than that of RLIS districts, which do not include the small districts 
that are eligible for SRSA funding. 

However, it is important to note that, due to the wide ranges of district sizes in subsets, average 
district sizes should be interpreted with caution, particularly in making comparisons. Lor 
example, in 2005-06, reported student population sizes in RLIS districts ranged from 56 to 
24,341 students, whereas districts nationwide ranged from 10 or fewer students to as many as 
1,014,058 students. Due to the widely varying sizes of districts, average district sizes should be 
interpreted with caution. 



Exhibit 18 

Mean District Student Popuiation, 2003-04 Through 2006-07 



3,145 3,150 3,156 3,186 




■ RLIS Districts ■ Non-RLIS Rural Districts □ All Districts 



Exhibit reads: In 2003-04, the mean student population in districts that 
received RLIS funds was 2,214. 

Source: Common Core of Data. 
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NCES Locale Designations 

The distribution of RLIS districts across NCES locale codes also remained relatively stable 
across the years of RLIS implementation (see Exhibit 19). The majority of RLIS districts had a 
NCES locale code of either 6 or 7, with between 40 and 50 percent of districts falling into each 
category. About 12 percent of districts reported census locale codes of 8. 



Exhibit 19 

Nationai Center for Education Statistics Locaie Codes, 
Eiigibie RLIS Districts, by Number and Percentage, 2006-07 





200( 

N=1 


i-07 

,286 




Nnmber of 
RLIS Districts 


Percentage 


1 -Large City 


0 


0% 


2-Mid-Size City 


0 


0% 


3-Urban Fringe of Large City 


1 


<1% 


4-Urban Fringe of Mid-Size City 


0 


0% 


5-Large Town 


1 


<1% 


6-Small Town 


536 


42% 


7-Rnral Ontside CBSA/MSA 


589 


46% 


8-Rnral Inside CBSA/MSA 


159 


12% 



Exhibit reads: In 2006-07, there were no Rural and Low-Income School (RLIS) districts 
with an NCES locale code^ of 1 . 

NCES categorizes local education authorities based on data provided by the Census. The 
locale codes included in this analysis are 6, 7, and 8. NCES locale code 6 is defined as 
“An incorporated place or Census-designated place with a population less than 25,000 
and greater than or equal to 2,500 and located outside a CBSA or MSA.” (CBSA or MSA 
stands for Core Based Statistical Area or Metropolitan Statistical Area.) NCES locale 
code 7 is defined as “Any territory designated as rural by tbe Census Bureau that is 
outside a CBSA or MSA of a Earge or Mid-size City.” NCES locale code 8 is defined as 
“Any territory designated as rural by tbe Census Bureau that is within a CBSA or MSA 
of a Earge or Mid-size City.” 

Note: As shown in Exhibit 5, two REIS districts have locale codes other than 6, 7 or 8. 
According to program staff members, these exceptions are likely the result of an anomaly 
in how two school districts are assigned locale codes in one state. 

Source: Common Core of Data. 
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Region of the Country 

Across the three school years of data, the distribution of RLIS districts across regions of the 
country remained relatively stable. In 2006-07, the majority of RLIS districts, about 69 percent, 
were located in the South. The remaining RLIS districts were distributed roughly evenly across 
the remaining regions: the West, Midwest, and Northeast (see Exhibit 20). 



Exhibit 20 

Regionai Distribution of RLIS Districts, 2006-07 




Exhibit reads: In 2006-07, most RLIS districts (69 percent) are in the Southern region of 
the United States. The remaining 31 percent of RLIS districts were spread relatively evenly 
across the Northeast, Midwest, and West. 

Source: Common Core of Data. 
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The plurality of U.S. districts overall, about 36 percent, were located in the Midwest. About 20 
percent of districts were located in each of the remaining regions. Compared with U.S. districts 
overall and non-RLIS rural districts, a substantially higher percentage of RLIS districts were 
located in the South and lower percentages of RLIS districts were located in the remaining 
regions (see Exhibit 21). 



Exhibit 21 

Regionai Distribution of U.S. Districts, by Percentage, 2006-07 




Exhibit reads: In 2006-07, 1 1 percent of RLIS districts were located in the 
Midwest. 

Source: Common Core of Data. 
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Student-Teacher Ratio 



From 2003-04 to 2006-07, average student-teacher ratios in RLIS districts decreased slightly 
from 14.7 to 14.2 over the four years. In 2006-07, the average student-teacher ratio for RLIS 
districts (14.0) was similar to that of all U.S. districts (14.2). In contrast, the average ratio for 
non-RLIS rural districts was 13.0, lower than the averages for both RLIS districts and districts 
nationally (see Exhibit 22). 



Exhibit 22 

Mean Student-Teacher Ratios, U.S. Districts, 
by Type of District, 2003-04 Through 2006-07 




2003-04 2004-05 2005-06 2006-07 



■ RLIS Districts ■ Non-RLIS Rural Districts □ All Districts 



Exhibit reads: In 2003-04, the mean student-teacher ratio in RLIS 
districts was 14.7. 

Source: Common Core of Data. 
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Levels ofPer-Pupil Spending 

Average per-pupil spending increased in RLIS districts between the 2003-04 and the 2006-07 
school years, from $8,435 to $9,842. In 2004-05, average per-pupil spending in RLIS districts 

o 

was lower than the average of all U.S. districts and the average of non-RLIS rural districts. 
Total expenditures in RLIS districts were just under $10,000 in 2006-07, compared with more 
than $12,000 for the same year in other rural districts and districts nationally (see Exhibit 23). 
As the RLIS program is intended to help rural school districts that serve students from low- 
income families, the RLIS program is targeting districts as intended 



Exhibit 23 

Average Per-Pupii Spending, U.S. Districts, 
by Type of District, 2003-04 Through 2006-07 




■ RLIS Districts □ Non-RLIS Rural Districts □ All Districts 



Exhibit reads: In 2003-04, the average per-pupil spending in RLIS districts was $8,435. 
Source: Common Core of Data. 



* These differences in per-pupil expenditures are partially explained by the fact that RLIS districts are more likely to 
be found in the South, where per-pupil expenditures tend to be lower than in the rest of the U.S. In addition, per- 
pupil expenditures in RLIS districts were lower than those in other districts in each region. Exhibit B-1 1 and Exhibit 
B-12 in Appendix B report average per-pupil expenditures by region and district type. 



Chapter 3 



39 







Ethnic and Racial Make-Up of Student Body 

From the 2003-04 school year through the 2006-07 school year, the racial composition of RLIS 
districts remained relatively stable. The majority of students in RLIS districts were white. About 
one-quarter of students in RLIS districts were black and about one-tenth of students in RLIS 
districts were Hispanic. Fewer than 5 percent were American Indian and about 1 percent were 
Asian (see Exhibit A- 17). 

In RLIS districts, the percentage of students representing certain racial or ethnic groups varied 
compared with U.S. districts overall and compared with non-RLIS rural districts. In 2006-07, for 
example, across all three types of districts, the majority of students were white, though RLIS 
districts reported a higher proportion of black students (23 percent) compared with non-RLIS 
rural districts (7 percent) and U.S. districts overall (17 percent). There was also a slightly higher 
proportion of Hispanic students in RLIS districts (II percent) compared with non-RLIS rural 
districts (9 percent). However, both RLIS and non-RLIS rural districts had lower proportions of 
Hispanic students than in U.S. districts overall (22 percent). RLIS districts served a similar 
percentage of Asian or Pacific Islander students as other rural districts (1 percent) but a lower 
percentage than U.S. districts overall (5 percent). Conversely, RLIS districts served higher 
percentages of American Indian or Alaskan Native students (4 percent) than other rural districts 
(2 percent) and U.S. districts overall (1 percent) (See Exhibit 24).^ 



^ An additional category of “Unknown” is included in this analysis to account for discrepancies between counts for 
racial or ethnic categories and district totals. As previously mentioned, the 2005-06 and 2006-07 files did not 
include values for total students in the district. Totals were therefore calculated by summing the counts by grade. It 
was noted that the sum of the racial or ethnic categories was slightly less than the sum of the grades, suggesting 
there were some students not accounted for in the racial or ethnic categories. Therefore, the “Unknown” category 
was added to account for these uncategorized students. 
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Exhibit 24 

Ethnic and Raciai Make-Up of Student Body, 
by Percentage and Type of District, 2006-07 



100% 



80% 




White Black Hispanic Asian/Pacific American 

Islander Indian/Alaskan 
Native 



■ RLIS Districts ■ Non-RLIS Rural Districtsn All Districts 



Exhibit reads: In 2006-07, white students constituted an average of 61 percent 
of the students in RLIS districts. 

Source: Common Core of Data. 





Proportion of Students Who Qualify for Free or Reduced-price Meals 

As expected, RLIS districts overall had a relatively large proportion of students who qualified for 
free or reduced-price meals, even when compared with the other types of rural districts. More 
than 60 percent of students in RLIS districts qualified for free or reduced-price meals over the 
four-year period, with that percentage remaining relatively stable from 2003-04 to 2006-07. 
Compared with U.S. districts overall and non-RLIS rural districts, higher percentages of students 
in RLIS districts qualified for free or reduced-price meals. From 2003-04 to 2006-07, the 
percentage of students that qualified for free or reduced-price meals in other types of districts 
also remained relatively stable (see Exhibit 25). 



Exhibit 25 

Average Percentage of Students Who Quaiify for Free or Reduced-price Meais, 
by Percentage and Type of District, 2003-04 Through 2006-07 



O 20% 

o 



5 0% 



I RLIS Districts ■ Non-RLIS Rural Districts □ All Districts 



Exhibit reads: In 2003-04, an average of 62 percent of students in RLIS 
districts qualified for free or reduced-price meals. 

Source: Common Core of Data. 
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Special Populations 

Across all four years of data, about 5 percent of students in RLIS districts were identified as 
limited English proficient (LEP). Compared with REIS districts, U.S. districts overall had higher 
percentages of students identified as limited English proficient, although these students still only 
accounted for about 10 percent of students nationally. In contrast, slightly lower percentages of 
students in non-REIS rural districts, about 4 percent, than REIS districts were identified as 
limited English proficient (see Exhibit 26). 

Across ah four years of data, between 13 and 15 percent of students in ah three types of districts 
had an Individualized Education Program (lEP) (see Exhibit 26). These students with disabilities 
require specialized instruction under the Individuals with Disabilities Education Act {IDEA). 
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Exhibit 26 

Average Proportion of Limited Engiish Proficient Students and 
Students with Individuaiized Education Programs, by Percentage and Type of 

District, 2003-04 Through 2006-07 

LEP Students Students with lEPs 




Exhibit reads: In 2003-04, an average of 5 percent of students in RLIS districts were limited English 
proficient. In that same year, an average of 15 percent of students in RLIS districts had an lEP. 

Source: Common Core of Data. 
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Chapter 4: Student Achievement in RLIS Districts 



Increasing student achievement and assisting districts in making AYP are primary goals of the 
RLIS program. This chapter examines student achievement in RLIS districts. First, a descriptive 
analysis of how RLIS districts were faring under the AYP provisions of NCLB in the 2007-08 
school year is provided. Next, trends in student achievement from 2002-03 through 2007-08 are 
examined, comparing trends in achievement in RLIS districts to trends in non-RLIS rural 
districts as well as the relationship between trends in student achievement and the number of 
years a district has received RLIS funds. Finally, in a sample of RLIS districts, the relationship is 
examined between student achievement and three characteristics of RLIS implementation: 

(a) priorities guiding district use of RLIS funds; (b) activities implemented using RLIS funds; 
and (c) district reports of the helpfulness of state technical assistance, activities implemented 
using RLIS funds. 

For each of the analyses presented in this chapter, the data sources used in the analysis are 
presented first, followed by a brief description of the analytic approach employed and finally, the 
findings. Additional information about the analytic approach and supporting documentation can 
be found in Appendix B. 

Key Findings 

Key findings on student achievement in RLIS districts include the following: 

• Overall, 54 percent of RLIS districts reporting data made AYP in 2007-08. 

• Among racial and ethnic subgroups, RLIS districts had the most difficulty making AYP 
for the black student subgroup, although the majority of RLIS districts with the minimum 
number of black students made AYP for this group. 

• Among all subgroups, RLIS districts had the most trouble making AYP for students with 
Individualized Education Programs (lEPs). 

• Analyses suggest that, across the six years, mathematics and reading achievement 
increased at a faster rate in REIS districts than in non-REIS rural districts, controlling for 
several district-level covariates. 

• There was no systematic relationship between the number of years a district received 
REIS and gains in student achievement. All REIS dosage groups — districts receiving one, 
two, three, four, five, and six years of REIS funding — exhibited positive gains relative to 
non-REIS districts though not all of these gains were statistically significant. 

• Exploratory analyses suggest that districts that set REIS goals to decrease the dropout 
rate and increase the quality of instruction had significantly higher mathematics 
achievement than REIS districts that did not set these goals. However, there was neither a 
relationship between a reported focus on specific activities and mathematics nor reading 
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achievement. There was no relationship between any district-identified RLIS goals and 
reading achievement. 



AYP Status in RLIS Districts 

This section presents data on the progress of RLIS districts in making AYP during the 2007-08 
school year, the most recent year for which data on AYP were available. Extant data for this 
analysis were drawn from the EDEN-EDFacts data system. For the 2007-08 school year, 93 
percent of REIS districts and 78 percent of all districts reported AYP data. (EDEN-EDFacts is 
still a relatively new data system, so data were not available for all districts.) Similar to the 
analyses presented in the previous chapter, several districts did not report values for some of the 
variables. For the 2007-08 school year, roughly 29 percent of districts submitted data on AYP 
for all subgroups. When they did not report information, these districts were not included in the 
analyses. Exhibit A-1 in Appendix A reports the total number of districts included in the data file 
for each school year. The number of districts that were missing values for the variables used in 
specific analyses can be determined by comparing the N reported for the analysis from the 
appropriate total number of districts reported in Exhibit A-1. 



Subgroups for Which AYP was Calculated 

All districts are not held accountable for making AYP for each subgroup under ACLfi. Each state 
is responsible for setting a minimum sample size for a subgroup to be included in AYP 
calculations. This is intended to reduce the chance that a district will be penalized due to 
statistical error as a result of small sample size as well as respect the privacy of individual 
students. 

In 2007-08, the number of subgroups for which AYP was calculated in REIS districts ranged 
from zero to eight. The highest percentages of REIS districts, a little over 20 percent, had three 
or four subgroups for which AYP was calculated (see Exhibit 27). 
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Exhibit 27 

Percentage of RLIS Districts by Number of Student Subgroups 
for Which AYP Was Caicuiated, 2007-08 




012345678 



Number of Applicable Subgroups 

Exhibit reads: In 2007-08, 8 percent of RLIS districts had zero applicable subgroups for 
which AYP was calculated in mathematics and reading. 

Source: EDEN-EDEacA. 





AYP Status 



To make AYP a district must meet targets for the percentage of students who are at or above 
state proficiency standards in math and reading overall and for all NCLB subgroups for which 
they are held accountable. Overall, 54 percent of RLIS districts made AYP in 2007-08. The vast 
majority of RLIS districts met AYP targets in mathematics (98 percent) and reading (98 percent) 
for white students. For minority students, many RLIS districts had no students or too few 
students in the category to report achievement data. For example, 46 percent of RLIS districts 
had too few black students to be held accountable for making AYP, while 79 percent of districts 
had too few Asian or Pacific Islander students. For those districts that did have a measurable 
number of minority students on whom to report, most made AYP. RLIS districts had the most 
difficulty making AYP for the black, non-Hispanic subgroup, although the majority of RLIS 
districts with the minimum number of students made AYP for this group (see Exhibits 28 and 
29). 



Exhibit 28 

Mathematics Proficiency Status by Ethnic and Raciai Group, 
by Percentage, RLIS Districts, 2007-08 




White Black Hispanic Asian/Pacific American 

(N=1.016) (N=542) (N=444) Islander Indian/Alaskan 

(N=168) Native 

(N=258) 

I ■ Met □ Not Met 



Exhibit reads: In 2007-08, 98 percent of RLIS districts made AYP in mathematics for 
white students. 1,016 RLIS districts reported AYP results and were held accountable for the 
white student subgroup. 

Source: EDEN-EDFaci5. 
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Exhibit 29 

Reading Proficiency Status by Ethnic and Raciai Group, 
by Percentage, RLIS Districts, 2007-08 

100% 



80% 

60% 

40% 

20% 

0% 



Exhibit reads: In 2007-08, 98 percent of RLIS districts made AYP in reading for white 
students. 1,016 RLIS districts reported AYP results and were held accountable for the white 
student subgroup. 

Source: EDEN-EDEacA. 



2% 3% 




White Black Hispanic Asian/Pacific American 

(N=1.016) (N=542) (N=444) Islander Indian/Alaskan 

(N=168) Native 

(N=258) 



■ Met □ Not Met 






RLIS districts had varied success meeting AYP targets for students with disabilities, 
economically disadvantaged students, and limited English proficient students. The majority of 
RLIS districts met AYP targets for economically disadvantaged (82 percent) and limited English 
proficient (83 percent) students. About 60 percent of RLIS districts met AYP targets in 
mathematics and reading for students with disabilities (see Exhibits 30 and 31). 



Exhibit 30 

Mathematics Proficiency Status by Speciai Popuiations, 
by Percentage, RLIS Districts, 2007-08 
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Students with Disabilities Economic Disadvantaged Limited English Proficient 
(N=873) (N=1,126) (N=264) 



I Met □ Not Met 



Exhibit reads: In 2007-08, 60 percent of RLIS districts made AYP in mathematics for 
students with disabilities. Eight-hundred and seventy-three RLIS districts reported AYP 
results and were held accountable for the students with disabilities subgroup. 

Source: EDEN-EDEact5. 







Exhibit 31 

Reading Proficiency Status by Speciai Popuiations, 
by Percentage, RLIS Districts, 2007-08 




Students with Disabilities Economic Disadvantaged Limited English Proficient 
(N=873) (N=1,126) (N=264) 



■ Met □ Not Met 



Exhibit reads: In 2007-08, 58 percent of RLIS districts made AYP in reading for 
students with disabilities. 873 RLIS districts reported AYP results and were held 
accountable for the students with disabilities subgroup. 



Source: EDEN-EDEacA. 






Exhibit 32 presents the percentage of districts that have a sufficient number of students in a 
subgroup that did not make AYP in either math or reading. For those REIS districts held 
accountable for certain student subgroups in 2007-08, the greatest percentage of districts, about 
50 percent, were not able to make AYP for the students with disabilities subgroup. About one- 
quarter of districts did not meet AYP for the black, limited English proficient, and economic 
disadvantaged subgroups. Eess than 5 percent of REIS districts did not meet AYP for the white 
and Asian/Pacific Islander subgroups. 



Exhibit 32 

Percentage of RLIS Districts Heid Accountabie for Student Subgroups 
That Did Not Make AYP for That Subgroup, 2007-08 

American Indian/Alaskan Native 
(N=258) 

Asian/Pacific Islander 
(N=168) 

Hispanic 
(N=444) 

Black 
(N=542) 

White 
(N=1.016) 

Students with Disabilities 
(N=873) 

Limited English Proficient 
(N=264) 

Economic Disadvantaged 
(N=1,126) 

0% 20% 40% 60% 

Exhibit reads: In 2007-08, 1 1 percent of RLIS districts held accountable for the American 
Indian/ Alaskan Native subgroup did not make AYP for that subgroup. 

Source: EDEN-EDFacf5. 
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Longitudinal Modeling of Student Achievement 

Six years of achievement data (2002-03 through 2007-08) were analyzed to answer the 
following research question: What are the trends in achievement in RLIS districts compared with 
other rural districts? A longitudinal analysis was carried out comparing results on state 
achievement tests in RLIS districts and non-RLIS rural districts. The longitudinal analysis is 
intended to provide a descriptive picture of trends in student achievement in reading and math in 
RLIS and non-RLIS districts holding district demographics, such as percent poverty, percent of 
students who qualify for special education services, and percent of limited English proficient 
students, constant. Two types of models were fit to the data. The first set of models compares 
gains in student achievement over time in RLIS and non-RLIS districts — across the six years of 
data. The second set of models examines the role dosage played in achievement gains. The RLIS 
districts were divided into dosage groups based on how many years they received RLIS funding 
between 2002-03 and 2007-08 and gains in each dosage group were compared with gains in 
non-RLIS districts. It is important to note that the study design and analysis do not support a 
conclusion that the RLIS program caused these differences in achievement. Observed differences 
could be a result of the RLIS program or a result of other non-program factors, such as a 
regression to the mean. 



Data Specifications 

Data analyses included only rural districts, those with an NCES locale code of 6, 7, or 8. This 
information was obtained from the CCD. Data on state designation (linking states to districts) 
also came from the CCD. 

Achievement data for the first three years came from NESESASD and for the last three years 
from EDEN/EDEact5. Data on achievement in mathematics and reading from EDEN-EDEacts 
were supplied in the form of district-le\el percent proficient on statewide assessments. Data from 
NESESASD were supplied in the form of grade level proficiency. Using all grades for which 
there were data, overall district-level proficiency was calculated from this data set for each year. 
Because state assessments are not comparable from state to state, the proficiency levels were 
converted to z-scores to assess the relative ranking of REIS districts within their states. 
Interpretation of z-scores is discussed below. 

Data used as covariates came from REIS program data, CCD, and EDEN/EDEact5. These 
covariates took the value of the average for a district over five years: 2002-03, 2003-04, 2004- 
OS, 2005-06, and 2006-07.^^ The covariates included in the longitudinal modeling were: average 
percent poverty, average percent minority, average percent of students with disabilities, average 
percent limited English proficiency, and average number of students per district. Exhibit 33 
summarizes the data sources used in the analyses. 



See Appendix B for details on calculating z-scores. 

" The 2002-03 data were missing the poverty percentages from the EDEN/EDFacfs data. The 2003-04 data did not 
have poverty percentages for non-RLIS districts. 

The use of aggregate values in a longitudinal analysis is problematic when values are likely to change 
substantially over a short period of time. However, in this analysis, that was not the case. Although the values for the 
actual covariates were sometimes statistically different across the time points, the actual differences were very small. 
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Exhibit 33 

Data Sources for Achievement Anaiyses by Schooi Year 



School 

Year 


CCD 


RLIS 

Eligibility 


RLIS Poverty 
Rating 


EDEacts-EDEN 

Achievement 


NLSLSASD 

Achievement 


2002-03 


X 


X 






X 


2003-04 


X 


X 


X (RLIS only) 




X 


2004-05 


X 


X 


X 




X 


2005-06 


X 


X 


X 


X 




2006-07 


X 


X 


X 


X 




2007-08 




X 


X 


X 




2008-09 




X 


X 


X 





Exhibit reads: Data used for the 2002-03 school year were drawn from the CCD, RLIS 
Eligihility data, and NLSLSASD Achievement Data. 



Determination of RLIS Status 

RLIS eligibility is determined annually. As a result, some districts move in and out of the 
program over the course of the period that we studied. For the most part, the data indicated a 
fairly steady number of RLIS-eligible districts. To ensure the robustness of the findings to these 
changes in RLIS status, the first set of longitudinal models was fit with RLIS status designated in 
one of three ways: 

• First, a district was classified as an RLIS district if the data from the program office 
indicated that the district was ever eligible to receive RLIS funds. 

• Next, a district was classified as an RLIS district only if it was eligible to receive funding 
in every year. 

• Finally, a district’s RLIS status was treated exactly as indicated by the data, changing 
from year to year. 

These specifications did not change the significance of the treatment effect estimates. Therefore, 
final interpretation was based on the third model that used the actual RLIS assignment. 



Missing Data 

Data were collected from a variety of sources, all of which had some missing data. When 
missing data could be substituted from one source to another (for example locale codes were 
found in the EDEN-EDFacts data as well as the Common Core Data) it was. The extent of the 
missing data is described in Appendix B. However, there were still missing data across the 
outcome measures as well as covariates. Since REIS eligibility is determined based on poverty 
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level, the relationship between percent of data missing, poverty levels, and achievement scores 
was examined. 



Overall, the data sets (one for mathematics and one for reading) included all variables of interest 
across all years for approximately 55 percent of districts. For mathematics, data were available 
for approximately 68 percent of RLIS districts and 55 percent of non-RLIS districts. For reading, 
data were available for approximately 65 percent of RLIS districts and 53 percent of non-RLIS 
districts. For the districts with achievement data, approximately 3 percent of the data were 
missing a value for average poverty level. In both of these data sets, achievement was slightly 
lower in the districts for which there were no poverty data. Data availability exhibits a similar 
pattern in RLIS and non-RLIS districts. In both the math and the reading data sets, there is less 
than 1 percent of poverty data missing for the RLIS districts and approximately 4 percent of the 
data missing for the non-RLIS districts. 

In comparing the districts that have and do not have achievement data, there are virtually no 
differences in terms of poverty. For both the math and reading data sets, those districts with data 
have an average poverty level of 10 percent and those without have an average poverty level of 9 
percent. Breaking down the data by condition, the pattern of data was consistent across districts 
with and without data, with the RLIS districts in both groups having a higher average poverty 
level (about 19 percent) as compared with the non-RLIS rural districts (about 8 percent). 

Overall, although there is a substantial amount of missing data, the pattern of missingness does 
not seem to be biased toward either RLIS or non-RLIS districts, nor toward districts with lower 
or higher levels of poverty. However, caution must be taken when interpreting the findings given 
the overall amount of missing data. 



Limitations 

In addition to the missing data, two other limitations of the analyses should be taken into 
account. First, the analysis relies solely on data aggregated at the district level rather than 
individual student or school level. As a result, the longitudinal analysis looks at change in district 
performance over time but cannot account for any compositional changes in the student 
populations in these districts. Second, the analysis relies on percent proficient to measure 
achievement rather than scale scores on an assessment. As a result, the analyses only provide 
information about achievement around the proficiency cut point but do not provide information 
about changes in performance among high- or low-performing students. 



Interpreting Impact Estimates 

The outcomes in the longitudinal data analysis are z-scores, which indicate a district’s relative 
standards compared with other districts in its states. For example, a district with a z score of 0.5 
would have a portion of students meeting proficiency standards that was half a standard 
deviation about the state’s mean. A change in a district’s z-score does not indicate a 
developmental increase in its achievement, such as a five-point increase in the portion of 
students making AYP. Instead, a gain in z-score means that the district moved within the 
distribution of districts in the state. 
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In addition, the estimates reported below are annual gain scores. They should be interpreted as 
the average growth between two school years. To estimate the average growth over the entire 
six-year period, the estimates should be multiplied by five. 



Approach to Modeling Student Achievement 

Achievement data were analyzed by fitting a series of multilevel models. This modeling 
approach takes into account the variability within groups (i.e., time periods and districts) prior to 
estimating gains across groups. This analytic method results in more precise estimates of 
predictor variables (and covariates) than a non-nested modeling approach. Each model was 
conceived as having two levels with time (level 1) nested within district (level 2). In the first set 
of models, the REIS treatment indicator (i.e., a dummy variable indicating whether a district was 
in the REIS program or not) was treated as a level 1 variable so that REIS status could change 
with time. Several district- level covariates were included as level 2 variables. In the second set 
of models, REIS dosage (number of years receiving REIS funds, ranging from zero to six) was 
modeled at the district level along with the same set of district-level controls. 



Random effects coefficients were included in both levels of the modeling. 
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Mathematics Findings 



• The annual rate of change in mathematics achievement was greater in RLIS than non- 
RLIS rural districts. However, the magnitude of this difference was relatively small (0.03 
standardized units). 

• There was no systematic relationship between the number of years a district received 
RLIS and gains in mathematics achievement. All RLIS dosage groups — districts 
receiving one, two, three, four, five, and six years of RLIS funding — exhibited positive 
annual gains relative to non-RLIS districts. However, these differences were statistically 
significant for districts that received funding for one, three, five, or six years. 

There was a significant difference in mathematics achievement between RLIS and non-RLIS 
rural districts at the initial time point of measurement (2002-03). In 2002-03, achievement in 
RLIS districts was 0.07 standardized units lower than achievement in non-RLIS rural districts. 
The annual rate of change in achievement was also significantly different between RLIS and 
non-RLIS rural districts. On average, mathematics achievement in RLIS districts increased 0.03 
standardized units more than that in non-RLIS rural districts each (see Exhibit 34). Over the six- 
year period, this represents an increase of 0.15 standardized units (0.03 for five years). It is 
important to note that the cause of the difference in the rate of change between RLIS and non- 
RLIS rural districts is not known. The difference could have been the result of the program or it 
could have been a result of non-program factors such as regression to the mean. In addition, the 
magnitude of the difference in growth is relatively small. By comparison, a 1 percentage point 
increase in the percent of students below the poverty line is associated with a -0.01 standardized 
unit change in mathematics achievement (see Exhibit B-1 1). 



Exhibit 34 

Mathematics Annuai Achievement Gains for RLIS Districts, 
2002-03 Through 2007-08 





Estimate 


Difference in achievement between REIS and non-REIS districts 
in 2002-03 


-0.07* 


Difference in gains in REIS and non-REIS districts 


0.03* 



Exhibit reads: Mathematics achievement in REIS districts increased 0.03 
standardized units more than that in non-REIS rural districts, which is 
statistically significant. 

* indicates statistical significance at the .05 level. 

Source: EDEN-EDFacts data, NESESASD, CCD. 
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The above analysis examines the relationship between receiving RLIS funds on academic 
achievement over time; however, it does not take into account the number of years that districts 
received that funding. The magnitude of the differences in gains across dosage groups (ranging 
from 0.02 to 0.04) was roughly similar to the difference between all RLIS and non-RLIS districts 
in mathematics achievement (0.03). Over the six-year period, this represents an increase of 0.10 
to 0.20 standardized units. This pattern of results suggests no systematic relationship between 
dosage and mathematics achievement gains. 



Exhibit 35 

Mathematics Annuai Achievement Gains for RLIS District, 
by Number of Years of RLIS Eligibility, 2002-03 Through 2007-08 



RLIS District 


Estimate 


Non-REIS rural (n = 26,392) 


-0.01* 


REIS -eligible for 1 year (n = 2,016) 


0.02* 


REIS -eligible for 2 years (n = 746) 


0.02 


REIS -eligible for 3 years (n = 476) 


0.04* 


REIS -eligible for 4 years (n = 739) 


0.02 


REIS -eligible for 5 years (n = 1,043) 


0.04* 


REIS-eligible for 6 years (n = 3, 118) 


0.03* 



Exhibit reads: The rate of aeademic achievement in non-RLIS rural districts was -0.01. 
This was significant at the .05 level. 

* indicates statistical significance at the .05 level. 

Source: EDEN-EDFacts data, NESESASD, CCD. 



Chapter 4 



59 







Reading Findings 



• The annual rate of change in reading achievement was greater in RLIS than non-RLIS 
rural districts. Again, the magnitude of this difference is small (.02 standardized units). 

• There was no systematic relationship between the number of years a district received 
RLIS and annual gains in reading achievement. All RLIS dosage groups — districts 
receiving one, two, three, four, five, and six years of RLIS funding — exhibited positive 
gains relative to non-RLIS districts. However, these differences were not statistically 
significant. 

There was a significant difference in reading achievement between RLIS and non-RLIS rural 
districts at the initial time point of measurement (2002-03). In 2002-03, achievement scores in 
RLIS districts were 0.07 standardized units lower than achievement scores in non-RLIS rural 
districts. The annual rate of change in achievement was significantly different between RLIS and 
non-RLIS rural districts. On average, reading achievement in RLIS districts increased 0.02 
standardized units more than that in non-RLIS rural districts (see Exhibit 36). Over the six-year 
period, this represents a change of 0.10 standardized units (0.02 for five years). By comparison, a 
1 percentage point increase in the percent of students below the poverty line is associated with a 
-0.01 standardized unit change in reading achievement (see Exhibit B-14). Again, as this 
evaluation did not examine causality, it cannot be ascertained whether this change is due to the 
REIS program or other, non-program factors. 



Exhibit 36 

Reading Annuai Achievement Gains for RLIS Districts, 
2002-03 Through 2007-08 





Estimate 


Difference in achievement 
between REIS and non-REIS 
districts in 2002-03 


-0.05* 


Difference in gains in REIS and 
non-REIS districts 


0.02* 



Exhibit reads: Reading achievement in REIS districts increased 0.02 standardized 
units more than that in non-REIS rural districts, which is statistically significant. 

* indicates statistical significance at the .05 level. 

Source: EDEN-EDFacts data, NESESASD, CCD. 
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The next analysis differentiates gains based on the number of years a district received funding by 
comparing each dosage amount to those districts not receiving any funding. Differences were not 
statistically significant, suggesting no systematic relationship between dosage and reading 
achievement gains (see Exhibit 37). 



Exhibit 37 

Reading Annuai Achievement Gains for RLIS District, 
by Number of Years of RLIS Eiigibiiity, 2002-03 Through 2007-08 



RLIS District 


Estimate 


Non-REIS rural (n = 25,911) 


-O.OI* 


REIS-eligible for 1 year (n = 1,969) 


O.OI 


REIS-eligible for 2 years (n = 702) 


0.01 


REIS-eligible for 3 years (n = 458) 


0.02 


REIS eligible for 4 years (n = 709) 


O.OI 


REIS eligible for 5 years (n = 1,01 1) 


0.03 


REIS eligible for 6 years (n = 2,970) 


0.02 



Exhibit reads: The rate of change for non-RLIS rural districts was -0.01. This was 
significant at the .05 level. 

* indicates statistical significance at the .05 level. 

Source: EDEN-EDFacts data, NESESASD, CCD. 
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Characteristics of RLIS Implementation and Student Achievement 

Finally, drawing on data from the survey of a sample of RLIS districts, the relationship between 
characteristics of RLIS district implementation and district student achievement was 
investigated. Two key characteristics of RLIS implementation were examined: district-reported 
priorities guiding use of RLIS funds; and district-reported activities implemented using RLIS 
funds. As in the analyses described above, it should be stressed that these analyses are solely 
descriptive in nature and should not be interpreted as describing a causal relationship between 
characteristics of RLIS implementation and student achievement. 



Data Specifications 

These analyses examined the relationship between characteristics of implementation reported by 
a sample of RLIS districts in spring 2009 and student achievement in the most recent school year 
for which data were available, 2007-08. Data on the percent of students who were proficient in 
each district was acquired from the EDEN-EDFacts data system and used to calculate district z- 
scores to allow comparisons to be made across states. 

Two sets of items from the district survey were included in the analysis. The first set of items 
focused on whether districts identified each of the following goals as a focus in their use of REIS 
funds: 

• Increasing student achievement in a particular subject 

• Reducing the high school dropout rate 

• Improving the quality of instruction 

• Improving the ability of English language learners to achieve proficiency in English and 
reach high academic standards 

• Ensuring that all students will be educated in learning environments that are safe, drug- 
free, and conducive to learning 

• Addressing issues specific to rural location, e.g., retaining teachers, providing distance 
learning opportunities, etc. 

If a district rated a goal as a moderate or major focus in their use of REIS funds, the goals were 
considered a district goal. 

Next, the relationship between the activities that were a focus of each district’s use of REIS 
funds and student achievement was examined. The following activities were examined: 

• Teacher recruitment and retention, including the use of signing bonuses and other 
financial incentives 

• Teacher professional development 

• Education technology, including software and hardware 

• Parental involvement activities 

• Activities authorized under the Safe and Drug-Eree Schools Program 

• Activities authorized under Title I, Part A 

• Eanguage instruction for EEE or EEP students. 
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Again, if a district rated an activity as a moderate or major focus in their use of RLIS funds, the 
activity was considered a focus in the district. 



A variety of district-level controls were included in the models. First, past district performance 
was controlled for through the inclusion of z-scores from the 2002-03 school year. A variety of 
district characteristics, including district size and locale were included from the 2006-07 CCD 
data file. Given the relationship between RLIS dosage and achievement discussed above, the 
number of years a district had been in receipt of district funds was also included in the model. 
Finally, at the state level, region of the country was included as a control. 



Limitations 

These analyses should only be considered exploratory, examining whether relationships could 
exist between how districts report using their RLIS funds and student achievement. As such, 
several limitations should be noted. The analyses are not designed to draw causal conclusions. In 
addition, the data used in these analyses are cross sectional. That is, the analyses examine the 
relationship between district survey responses and the most recent year of achievement data 
(2007-08), though the surveys were implemented in the spring of the 2008-09 school year. As a 
result, the analyses cannot provide information about the direction of any relationship found. For 
example, if a statistically significant relationship is found between a district having a particular 
goal it could be the case that setting that goal led to increased student achievement, but it could 
also be the case that the district set that goal because of higher levels of student achievement, or 
a third factor could account for the results. 

Similar to the longitudinal analysis above, missing data are also an issue in this analysis. When 
all of the covariates or outcome data were not available, cases were not included in the analysis. 
These analyses only included the 467 RLIS eligible districts for which there was both survey 
data and student assessment data. In addition, the limitations of using a district-level z-score 
calculated from the percent of students who met proficiency standards discussed above hold for 
this analysis as well. 

Finally, the analyses presented in this chapter test multiple hypotheses. As a result, it is possible 
that we find statistical significance simply by chance. While multiplicity adjustments are not 
needed in purely exploratory analyses, this limitation should be noted and further analyses are 
needed before firm conclusions are drawn (Schochet, 2008). 



Approach to Modeling 

Given the nested structure of the data, a two-level hierarchical linear modeling approach was 
employed in these analyses. Each model was conceived as having two levels with district (level 
1) nested within state (level 2). The variables of interest (goals, activities, and ratings of state 
technical assistance) were included as level 1 predictors along with the controls discussed above. 
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Mathematics Findings 



• Districts that set RLIS goals to decrease the dropout rate and increase the quality of 
instruction had significantly higher mathematics achievement than districts that did not 
set these goals. None of the other goals set by RLIS districts were significantly associated 
with mathematics achievement. 

• District reports of a focus on particular activities using RLIS funds were not significantly 
related to mathematics achievement for any of the activities. 

Goals. Mathematics achievement in districts that reported that reducing the dropout rate was a 
focus of their use of RLIS funds was, on average, 0.3 standardized units higher than in districts 
that did not identify this as an RLIS goal (see Exhibit 38). Similarly, mathematics achievement 
in districts that reported using their RLIS funds to increase the quality of instruction was 1.0 
standard units higher than in districts that did not identify this as a goal. Both of these 
relationships were significant at the .05 level. Interestingly, the coefficient on a district having a 
student achievement goal is negative. While this relationship is not statistically significant, it 
reminds the reader that with cross-sectional data we do not know the directionality of the 
relationships examined. One plausible explanation for the results presented here is that districts 
with low student achievement are more likely to set student achievement goals, possibly 
explaining the positive relationship between some of the other goals and student achievement. 
This explanation is supported by the finding in chapter 2 that analyzing and discussing district 
data is often a part of planning for the use of RLIS funds. 



Exhibit 38 

Mathematics Achievement Associated With Certain RLIS Goais, 2007-08 



RLIS Goal 


Coefficient 


Student Achievement Goal 


-0.12 


Dropout Goal 


0.30* 


Instruction Goal 


1.00* 


ELL Goal 


0.03 


Safety Goal 


0.03 


Rural Goal 


-0.11 



Exhibit reads: Districts with a student achievement goal had mathematics 
achievement that was 0.12 standardized units lower on average. This relationship is 
not statistically significant. 

* indicates statistical significance at the .05 level. 

Source: EDEN-EDFacts data, NESESASD, CCD, REIS district survey. 
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Activities. District survey respondents were asked whether certain activities were a focus of their 
use of RLIS funds. There was no relationship between focusing on any of these activities and 
mathematics achievement (see Exhibit 39). 



Exhibit 39 

Mathematics Achievement 
Associated With Certain RLIS Activities, 2007-08 



RLIS Activity 


Coefficient 


Teacher Recmitment 


-0.08 


Professional Development 


-0.09 


Educational Technology 


0.16 


Parent Involvement 


0.01 


Safe and Dmg-Free Schools 


0.00 


Title I, Part A 


0.07 


Language Instmction for ELL 


0.00 



Exhibit reads: Districts that focused on teacher recruitment activities had 
mathematics achievement that was 0.08 standardized units lower on average. This 
relationship is not statistically significant. 

* indicates statistical significance at the .05 level 

Source: EDEN-EDFacts data, NESESASD, CCD, REIS district survey. 
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Reading Findings 



• District reading achievement was not significantly associated with districts reporting a 
focus on any of the RLIS goals. 

• Districts were asked which activities were a priority in the use of RLIS funds. None of 
these activities was significantly related to reading achievement in the sample of 
surveyed RLIS districts. 

Goals. District reading achievement was not significantly associated with districts specifying any 
of the RLIS goals (see Exhibit 40). 



Exhibit 40 

Reading Achievement 

Associated With Certain RLIS Goais, 2007-08 



RLIS Goal 


Coefficient 


Student Achievement Goal 


0.26 


Dropout Goal 


0.19 


Instmction Goal 


0.68 


ELL Goal 


-0.16 


Safety Goal 


0.21 


Rural Goal 


-0.15 



Exhibit reads: Having a student achievement goal was associated with reading 
achievement that was on average 0.26 standardized units higher. This relationship is 
not statistically significant. 

* indicates statistical significance at the .05 level. 

Source: EDEN-EDFacts data, NESESASD, CCD, REIS district survey. 
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Activities. Reading achievement was not significantly related to the implementation of any of the 
RLIS activities (see Exhibit 41). 



Exhibit 41 

Reading Achievement 

Associated With Certain RLIS Activities, 2007-08 



RLIS Activity 


Coefficient 


Teacher Recmitment 


-0.03 


Professional Development 


-0.04 


Educational Technology 


0.06 


Parent Involvement 


0.02 


Safe and Dmg-Free Schools 


0.17 


Title I, Part A 


-0.03 


Language Instruction for ELL 


-0.46 



Exhibit reads: Focusing RLIS funds on teacher recruitment was associated with 
reading achievement that was on average 0.03 standardized units lower. This 
relationship is not statistically significant. 

* indicates statistical significance at the .05 level. 

Source: EDEN-EDFacts data, NESESASD, CCD, REIS districts survey. 
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Chapter 5: Conclusions 



Surveys of state and district RLIS coordinators, as well as interviews with district RLIS 
coordinators and a close review of district documents, showed that, in addition to the primary 
goal of making AYP, the most common goals for district RLIS programs were improving the 
quality of instruction and improving student achievement in specific subject areas. Interview 
reports by district RLIS coordinators and a review of district documents indicated that districts 
primarily attempted to achieve these goals by using RLIS funds for teacher pay, educational 
technology, professional development, and materials; district and state survey respondents also 
reported that RLIS funds were frequently used for educational technology, as well as to support 
activities authorized under Title I, Part A. It appeared that states and districts took advantage of 
the flexibility of the RLIS program that allowed them to use these funds to meet specific needs in 
their districts. According to survey respondents, states provided useful and timely training and 
technical assistance to districts on the RLIS program, in particular, by providing assistance in 
identifying specific needs for improvement and how to focus RLIS funds on making AYP. 

Except for the nature of RLIS being the last of their federal grant allocations to be finalized, 
which may result in constraints on a district's budgeting process, neither the interviews nor the 
surveys with states and districts indicated the existence of major challenges to implementation of 
the RLIS program by states or districts. 

Characteristics of RLIS districts have remained relatively stable over the years of the program’s 
implementation. On average, RLIS districts have more students than other rural districts, but 
fewer students than districts nationally. Consistent with program goals, RLIS districts serve a 
higher proportion of students who qualified for free or reduced-price meals than districts overall 
or non-RLIS rural districts. Despite a slight increase in per-pupil spending in RLIS districts 
between 2003-04 and 2004-05, per-pupil spending in RLIS districts remained substantially 
lower than in all districts nationally or in other rural districts. 

Overall, 54 percent of RLIS districts met all AYP targets in mathematics and reading in 2007- 
08. Among all subgroups, RLIS districts had the most trouble making AYP for students with 
Individualized Education Programs (lEPs); among racial and ethnic subgroups, REIS districts 
had the most difficulty making AYP for the black student subgroup, although the majority of 
REIS districts with the minimum number of students made AYP for this group. 

Though achievement gains cannot be attributed to the REIS program, the rate of annual 
academic improvement in mathematics (0.03 standardized units) and reading (0.02 standardized 
units) for districts that received REIS funding was significantly greater than for non-REIS 
districts. Eurther research would need to be done to understand whether REIS funding is 
responsible for these gains in achievement as well as the mechanism through which such a 
relationship may operate. 



Chapter 5 



68 






69 




References 



U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, Common Core of Data: School Years 2003-04 and 2006-07, available at 
http://nces.ed.gov/ccd/, cited September 2009. 

U.S. Department of Education, Institute of Education Sciences (2008). Technical methods report: 
Guidelines for multiple testing in impact evaluations, by Peter Z Schochet, Washington, D.C.: 
Author. 

U.S. Department of Education (2010). National Eongitudinal School-Eevel State Assessment 
Score Database, 2002-05, previously unpublished tabulations. Available from 
http://38.112.57.50. 

U.S. Department of Education, Office of Elementary and Secondary Education (2009). 

Program data from the Office of Elementary and Secondary Education, Rural Education 
Achievement Program (REAP). 

U.S. Department of Education, Office of Elementary and Secondary Education (2010). 

Rural and Eow-Income Schools Program: Eligibility. Available from 
http://www2.ed.gov/programs/reaprlisp/eligibility.html, cited April 2010. 

U.S. Department of Education, Office of Planning, Evaluation and Policy Development (2010). 
EDEacts, previously unpublished tabulations. 



References 



70 






71 




APPENDIX A: SUPPORTING DATA FOR CHAPTER 3 

Demographic Data for RLIS, Non-RLIS Rural, and All U.S. Districts 

The tables below provide supporting data for the exhibits presented in Chapter 3 of the text. 
Additional demographic data are presented, along with the number of cases included in each 
figure. 



Exhibit A-1 

Totai Districts in CCD Data Fiies, 2003-04 Through 2006-07 





2003-04 


2004-05 


2005-06 


2006-07 


RLIS Districts 


1,299 


1,127 


1,188 


1,287 


All Districts 


17,521 


17,804 


17,940 


18,786 


Non-RLIS Rural Districts 


8,789 


8,835 


8,424 


8,306 



Exhibit reads: In 2003-04, 1,299 districts that received RLIS funds reported data; in 2004- 
OS, 1,127 districts reported data; in 2005-06, 1,188 districts reported data; in 2006-07, 1,287 
districts reported data. 

Source: Common Core of Data. 
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Exhibit A-2 

District Student Popuiation, 2003-04 Through 2006-07 





2003-04 




N 


Mean 


Max. 


RLIS Districts 


1,297 


2,214 


24,056 


All Districts 


15,335 


3,145 


1,041,976 


Non-RLIS Rural Districts 


8,492 


1,117 


40,382 





2004-05 




N 


Mean 


Max. 


RLIS Districts 


1,123 


2,150 


24,268 


All Districts 


15,391 


3,150 


1,032,485 


Non-RLIS Rural Districts 


8,560 


1,140 


41,205 





2005-06 




N 


Mean 


Max. 


RLIS Districts 


1,188 


2,228 


24,341 


All Districts 


15,435 


3,156 


1,014,058 


Non-RLIS Rural Districts 


8,423 


1,135 


42,035 





2006-07 




N 


Mean 


Max. 


RLIS Districts 


1,284 


2,204 


23,825 


All Districts 


15,334 


3,186 


999,150 


Non-RLIS Rural Districts 


7,807 


1,161 


42,572 



Exhibit reads: In 2003-04, 1,297 districts that received RLIS funds reported student 
population information. For those districts, the mean district size was 2,214 students, 
with a maximum of 24,056. 

Source: Common Core of Data. 






Exhibit A-3 

Comparison of Districts’ Faii Membership and Sum of Grade Levei Totais, 

2003-04 Through 2004-05 





2003-04 


2004-05 




Fall 

Membership 


Sum of 
Grade Level 
Totals 


Fall 

Membership 


Sum of 
Grade Level 
Totals 


RLIS Districts 


2,872,011 


2,863,106 


2,414,494 


2,403,049 


Non-RLIS Rural Districts 


9,487,085 


9,588,749 


9,759,788 


9,807,602 


All U.S. Districts 


48,221,532 


49,227,340 


48,477,478 


49,387,565 



Exhibit reads: In 2003-04, RLIS districts reported a total fall membership of 2,872,011. The 
sum of the grade totals for RLIS districts in 2003-04 was 2,863,106. 

Source: Common Core of Data. 






Exhibit A-4 

NCES Locaie Codes, RLIS Districts, 2003-04 Through 2006-07 





2003-04 

N=l,299 


2004-05 

N=l,124 


2005-06 

N=l,185 


2006-07 

N=l,286 




N 


Percentage 


N 


Percentage 


N 


Percentage 


N 


Percentage 


1 -Large City 


0 


0.0% 


0 


0.0% 


0 


0.0% 


0 


0.0% 


2-Mid-Size City 


6 


0.5% 


0 


0.0% 


0 


0.0% 


0 


0.0% 


3-Urban Fringe of 
Large City 


29 


2.0% 


0 


0.0% 


1 


0.1% 


1 


0.1% 


4-Urban Fringe of 
Mid-Size City 


42 


3.0% 


0 


0.0% 


0 


0.0% 


0 


0.0% 


5-Large Town 


5 


0.4% 


0 


0.0% 


1 


0.1% 


1 


0.1% 


6-Small Town 


500 


39.0% 


482 


43.0% 


500 


42.0% 


536 


42.0% 


7-Rnral Ontside 
CBSA/MSA 


571 


44.0% 


531 


47.0% 


555 


47.0% 


589 


46.0% 


8-Rnral Inside 
CBSA/MSA 


144 


11.0% 


111 


10.0% 


128 


11.0% 


159 


12.0% 



Exhibit reads: In 2003-04, no RLIS districts had a locale code of 1-Large City. 

Note: RLIS eligibility is determined by a district’s NCES locale code of two years earlier. This 
table presents district locale codes for the school year in which funds were received, which may 
explain the existence of RLIS districts with a locale code other than 6, 7, or 8. 

Source: Common Core of Data. 
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Exhibit A-5 

NCES Locaie Codes, U.S. Districts, 2006-07 





RLIS Districts 
N=l,286 


All Districts 
N=15,719 


Non-RLIS rural 
Districts 
N=8,306 




N 


Percentage 


N 


Percentage 


N 


Percentage 


1-Large City 


0 


0.0% 


940 


6.0% 


0 


0.0% 


2-Mid-Size City 


0 


0.0% 


976 


6.0% 


0 


0.0% 


3-Urban Fringe of Large City 


1 


0.1% 


2,583 


16.0% 


0 


0.0% 


4-Urban Fringe of Mid-Size City 


0 


0.0% 


1,497 


10.0% 


0 


0.0% 


5-Large Town 


1 


0.1% 


133 


1.0% 


0 


0.0% 


6-Small Town 


536 


42.0% 


1,687 


11.0% 


1,151 


14.0% 


7-Rural Outside CBS A/MS A 


589 


46.0% 


5,235 


33.0% 


4,646 


56.0% 


8-Rural Inside CBSA/MSA 


159 


12.0% 


2,668 


17.0% 


2,509 


31.0% 



Exhibit reads: In 2006-07, no districts that received RLIS funds had a locale code of 1 -Large 
City. Nationwide, 940 districts (6 percent) had a locale code of 1 -Large City, and no non-RLIS 
rural districts had a locale code of 1 -Large City. 

Source: Common Core of Data. 



Exhibit A-6 

Regionai Distribution of RLIS Districts, by Number and Percent, 2003-04 Through 

2006-07 




Exhibit reads: In 2003-04, 187 districts that received RLIS funds were located in the Midwest, 
about 14 percent of RLIS districts. In 2004-05, 135 districts that received RLIS funds were 
located in the Midwest, about 12 percent of RLIS districts. 

Source: Common Core of Data. 
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Exhibit A-7 

Regionai Distribution of U.S. Districts, by Number and Percent, 2006-07 





RLIS Districts 
N=l,285 


All Districts 
N=17,755 


Non-RLIS Districts, 
Rural 
N=8,078 




N 


Percent 


N 


Percent 


N 


Percent 


Midwest 


137 


11% 


6,369 


36% 


3,297 


41% 


Northeast 


119 


9% 


4,018 


23% 


1,401 


17% 


South 


883 


69% 


3,967 


22% 


1,692 


21% 


West 


146 


11% 


3,388 


19% 


1,688 


21% 



Exhibit reads: In 2006-07, 137 districts that received RLIS funds were located in the Midwest, 
about 1 1 percent of RLIS districts. 

Source: Common Core of Data. 






Exhibit A-8 

Student-Teacher Ratios, RLIS Districts, 2003-04 Through 2006-07 





2003-04 

N=l,289 


2004-05 

N=l,076 


2005-06 

N=l,173 


2006-07 

N=l,068 


Mean 


14.7 


14.5 


14.4 


14.2 


Standard Deviation 


2.3 


2.2 


2.2 


2.3 



Exhibit reads: In 2003-04, the average student-teacher ratio for RLIS districts was 
14.7 students per teacher, with a standard deviation of 2.3. 

Note: Due to a wide variance in reported student-teacher ratios, the sample was 
limited to those districts with ratios greater than 0 hut less than 50. 

Source: Common Core of Data. 



Exhibit A-9 

Student-Teacher Ratios, U.S. Districts, 2006-07 





RLIS Districts 
N=l,068 


All Districts 
N=13,232 


Non-RLIS 

Districts, 

Rnral 

N=6,628 


Mean 


14.2 


14.0 


13.0 


Standard Deviation 


2.3 


4.4 


4.0 



Exhibit reads: In 2006-07, the average student-teacher ratio for RLIS 
districts was 14.2 students per teacher, with a standard deviation of 2.3. 



Note: Due to a wide variance in reported student-teacher ratios, the sample 
was limited to those districts with ratios greater than 0 hut less than 50. 

Source: Common Core of Data. 
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Exhibit A-10 

Average Per-Pupii Spending, by Category, RLIS Districts, 2003-04 Through 

2006-07 





2003-04 

N=l,297 


2004-05 

N=l,123 


2005-06 

N=l,188 


2006-07 

N=l,284 


Elementary-Secondary 


$7,461 


$7,963 


$8,258 


$8,773 


Instrnction 


$4,523 


$4,804 


$4,944 


$5,261 


Snpport Services 


$2,525 


$2,723 


$2,856 


$3,030 


Other Elementary-Secondary 
Programs 


$414 


$436 


$458 


$481 


Salary 


$4,650 


$4,885 


$5,008 


$5,296 


Benefits 


$1,316 


$1,424 


$1,519 


$1,679 


Capital Ontlay 


$756 


$684 


$664 


$816 


Non Elementary-Secondary 
Programs 


$66 


$69 


$67 


$69 


Average Total Expenditnres 


$8,435 


$8,865 


$9,153 


$9,842 



Exhibit reads: In 2003-04, the average per-pupil spending on elementary-secondary programs 
in RLIS districts was $7,461. 

Source: Common Core of Data. 



Exhibit A-11 

Average Per-Pupii Spending, by Category, U.S. Districts, 



2006-07 





RLIS Districts 
N=l,284 


All Districts 
N=15,334 


Non-RLIS 
Rnral Districts 
N=7,807 


Elementary-Secondary 


$8,773 


$10,283 


$10,466 


Instrnction 


$5,261 


$6,108 


$6,238 


Snpport Services 


$3,030 


$3,759 


$3,773 


Other Elementary-Secondary 
Programs 


$481 


$417 


$456 


Salary 


$5,296 


$5,909 


$6,060 


Benefits 


$1,679 


$1,932 


$1,920 


Capital Ontlay 


$816 


$1,053 


$1,034 


Non Elementary-Secondary 
Programs 


$69 


$93 


$73 


Average Total Expenditnres 


$9,842 


$11,777 


$11,965 



Exhibit reads: In 2006-07, the average per-pupil spending on elementary-secondary programs 
in RLIS districts was $8,773. 

Source: Common Core of Data. 
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Exhibit A-12 

Average District Per-Pupii Spending by Region, 2006-07 





RLIS Districts 
N=l,284 


All Districts 
N=15,334 


Non-RLIS Rural 
Districts 
N=7,807 




N 


Mean 


N 


Mean 


N 


Mean 


Midwest 


137 


$9,611 


5,592 


$10,384 


3,180 


$10,262 


Northeast 


119 


$13,795 


3,119 


$15,204 


1,314 


$14,710 


South 


882 


$9,328 


3,530 


$10,404 


1,614 


$11,090 


West 


146 


$9,939 


3,093 


$12,406 


2,543 


$13,861 


U.S. Average 


1,284 


$9,842 


15,334 


$11,777 


7,807 


$11,965 



Exhibit reads: In RLIS districts in the Midwest, the average per-pupil spending was $9,611. 
Source: Common Core of Data. 



















































Exhibit A-13 

Ethnic and Raciai Make-up of Student Body, 
RLIS Districts, 2003-04 Through 2006-07 





2003-04 

N=l,259 


2004-05 

N=l,087 


2005-06 

N=l,188 


2006-07 

N=l,284 




Sum 


Percentage 


Sum 


Percentage 


Sum 


Percentage 


Sum 


Percentage 


White 


1,641,049 


57% 


1,344,630 


56% 


1,585,494 


60% 


1,732,794 


61% 


Black 


693,678 


24% 


577,175 


24% 


625,830 


24% 


659,440 


23% 


Hispanic 


291,358 


10% 


279,449 


12% 


301,571 


11% 


297,860 


11% 


Asian/ 

Pacific 

Islander 


15,401 


1% 


12,327 


1% 


14,882 


1% 


17,067 


1% 


American 

Indian/ 

Alaskan 

Native 


114,333 


4% 


93,927 


4% 


101,270 


4% 


106,063 


4% 


Unknown 


116,192 


4% 


106,986 


4% 


17,557 


1% 


13,054 


1% 


Fall 

Membership 


2,872,011 


100% 


2,414,494 


100% 


2,646,604 


100% 


2,826,278 


100% 



Exhibit reads: In 2003-04, there were 1,641,049 white students in RLIS districts, about 57 
percent of total students. 

Source: Common Core of Data. 






Exhibit A-14 

Ethnic and Raciai Make-up of Student Body, U.S. Districts, 2006-07 





RLIS Districts 
N=l,284 


Non-RLIS Districts, 
Rnral 
N=8,003 


All Districts 
N=16,062 




Sum 


Percentage 


Sum 


Percentage 


Sum 


Percentage 


White 


1,732,794 


61% 


7,306,209 


81% 


27,347,536 


56% 


Black 


659,440 


23% 


650,604 


7% 


8,275,666 


17% 


Hispanic 


297,860 


11% 


776,929 


9% 


10,482,519 


22% 


Asian/Pacific Islander 


17,067 


1% 


106,627 


1% 


2,279,868 


5% 


American Indian/ 
Alaskan Native 


106,063 


4% 


185,115 


2% 


588,953 


1% 


Unknown 


13,054 


1% 


86,961 


0.4% 


635,078 


1.3% 


Total Stndent Popnlation 


2,829,405 


100% 


9,066,256 


100% 


48,852,324 


100% 



Exhibit reads: In 2006-07 there were 1,732,794 white students in RLIS districts, about 61 
percent of total students. 

Source: Common Core of Data. 






Exhibit A-15 

Students Who Quaiify for Free or Reduced-price Meais, by Number and 
Percentage, U.S. Districts, 2003-04 Through 2006-07 





2003-04 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,122 


2,492,878 


1,539,817 


62% 


All Districts 


13,845 


43,611,863 


17,535,650 


40% 


Non-RLIS Rural 
Districts 


7,887 


8,644,497 


3,040,172 


35% 





2004-05 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,003 


2,137,532 


1,354,777 


63% 


All Districts 


13,978 


45,314,142 


18,135,119 


40% 


Non-RLIS Rural 
Districts 


8,072 


9,174,763 


3,332,410 


36% 





2005-06 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,184 


2,641,047 


1,639,082 


62% 


All Districts 


15,007 


47,485,701 


19,504,952 


42% 


Non-RLIS Rnral 
Districts 


7,925 


9,180,627 


3,386,428 


37% 





2006-07 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,281 


2,823,606 


1,759,291 


62% 


All Districts 


14,378 


46,864,994 


20,813,787 


43% 


Non-RLIS Rnral 
Districts 


7,428 


8,942,328 


3,335,477 


37% 



Exhibit reads: In 2003-04, 1,122 RLIS districts reported data on both total student population 
and students who qualify for free or reduced-price meals. In those districts, there were 
2,492,878 students total and 1,539,817 students who qualified for free or reduced-price meals, 
about 62 percent of students total. 



Source: Common Core of Data. 






Exhibit A-16 

Students Who Are Limited Engiish Proficient, U.S. Districts, by Number 
and Percentage, 2003-04 Through 2006-07 





2003-04 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,157 


2,641,166 


124,477 


5% 


All Districts 


12,923 


40,512,731 


3,819,470 


9% 


Non-RLIS Rural 
Districts 


7,397 


8,007,678 


297,330 


4% 





2004-05 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,024 


2,244,961 


110,384 


5% 


All Districts 


12,438 


39,318,402 


3,898,520 


10% 


Non-RLIS Rural 
Districts 


7,381 


8,178,601 


306,824 


4% 





2005-06 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,102 


2,481,684 


123,262 


5% 


All Districts 


13,431 


43,543,749 


4,208,593 


10% 


Non-RLIS Rural 
Districts 


7,084 


8,052,302 


306,315 


4% 





2006-07 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,013 


2,449,234 


98,405 


4% 


All Districts 


11,426 


35,905,927 


2,357,285 


7% 


Non-RLIS Rural 
Districts 


5,581 


7,332,289 


240,285 


3% 



Exhibit reads: In 2003-04, 1,157 RLIS districts reported data on both total student population and the 
number of Limited English Proficient students. In those districts, there were 2,641,166 students total 
and 124,477 students were LEP, about 5 percent of total students. 

Source: Common Core of Data. 






Exhibit A-17 

Students With Individuaiized Education Programs, 

U.S. Districts, by Number and Percentage, 2003-04 Through 2006-07 





2003-04 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,241 


2,795,666 


429,531 


15% 


All Districts 


14,614 


45,372,370 


6,149,728 


14% 


Non-RLIS Rural 
Districts 


8,184 


9,100,063 


1,313,385 


14% 





2004-05 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,039 


2,300,857 


353,549 


15% 


All Districts 


13,546 


43,371,565 


5,911,083 


14% 


Non-RLIS Rural 
Districts 


7,742 


8,960,685 


1,302,461 


15% 





2005-06 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,187 


2,645,673 


388,706 


15% 


All Districts 


15,419 


48,698,476 


6,622,178 


14% 


Non-RLIS Rural 
Districts 


8,131 


9,232,436 


1,300,255 


14% 





2006-07 




N 


Total 

Student 

Population 


Students 
Who Qualify 


Percentage 
of Students 
Who Qualify 


RLIS Districts 


1,253 


2,759,522 


406,851 


15% 


All Districts 


13,945 


46,168,379 


6,183,357 


14% 


Non-RLIS Rural 
Districts 


7,175 


8,673,409 


1,230,813 


14% 



Exhibit reads: In 2003-04, 1,241 RLIS districts reported data on both total student population and the 
number of students with Individualized Education Programs. In those districts, there were 2,795,666 
students total and 429,531 students had lEPs, about 15 percent of students total. 

Source: Common Core of Data. 






APPENDIX B: STUDENT ACHIEVEMENT TECHNICAL APPENDIX 



The appendix provides a description of the data used and the analysis techniques employed in 
Chapter 4 of this report. 

AYP Status in RLIS Districts 



Exhibit B-1 

Mathematics Proficiency Status by Ethnic and Raciai Group, 
RLIS Districts by Number and Percentage, 2007-08 





Not Applicable 
or Too Few 
Stndents for 
Reliability 


Not Met 


Met 
Because 
of Safe 
Harbor 


Met 


N 


White 


5.2% 


3.0% 


1.0% 


90.9% 


1,151 


Black 


45.6% 


12.3% 


6.5% 


35.6% 


1,047 


Hispanic 


55.5% 


3.2% 


0.6% 


40.8% 


1,033 


Asian/Pacific Islander 


78.9% 


0.5% 


0.1% 


20.5% 


824 


American Indian/Alaskan 
Native 


70.0% 


2.3% 


0.8% 


26.9% 


866 



Exhibit reads: In 2007-08, about 5.2 percent of RLIS districts had no or too few white students to 
report proficiency in mathematics and 3.0 percent of districts did not meet AYP targets for white 
students. 

Source: EDEN-EDCacA; National Eongitudinal School-Eevel State Assessment Score Database 
(NESESASD); Common Core of Data. 
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Exhibit B-2 

Reading Proficiency Status by Ethnic and Raciai Group, 
RLIS Districts by Number and Percentage, 2007-08 





Not Applicable 
or Too Few 
Students for 
Reliability 


Not Met 


Met 

Because of 
Safe 
Harbor 


Met 


N 


White 


5.7% 


1.7% 


0.3% 


92.4% 


1,077 


Black 


44.4% 


8.9% 


3.4% 


43.3% 


977 


Hispanic 


53.8% 


5.5% 


0.8% 


39.9% 


966 


Asian/Pacific Islander 


78.4% 


0.6% 


0.3% 


20.7% 


781 


American Indian/Alaskan 
Native 


68.4% 


2.9% 


0.5% 


28.2% 


817 



Exhibit reads: In 2007-08, about 5.7 percent of RLIS districts had no or too few white students to 
report proficiency in reading, and 1.7 percent did not meet AYP targets. 

Source: EDEN-EDEacA; National Eongitudinal School-Eevel State Assessment Score Database 
(NESESASD); Common Core of Data. 






Data Specifications for the Analyses of Student Achievement 



This section first discusses all common elements of the achievement analyses including: missing 
data; sample sizes; calculating proficiency ratings; computing z-scores; interpreting impact 
estimates; calculating covariates; testing covariates; assessing dosage; and data used in the 
analysis. 

Missing Data. Examination of the missing data revealed that approximately 55-56 percent of the 
raw data were complete for all variables included in the modeling. Exhibit B-3 shows these 
numbers. 



Exhibit B-3 

Number of Data Points by Subject Area With Data on Aii Covariates 



Subject 


Number of Data Points 


Percentage by 
Subject 


Math 


33,127 


56% 


Read 


32,342 


55% 



Exhibit reads: There were 33,127 data points for mathematics achievement 
(across six years of data) for which there were also data on all covariates. 
This number represents approximately 56 percent of all mathematics data. 

Source: EDEN-EDEact^; National Eongitudinal School-Eevel State 
Assessment Score Database (NESESASD); Common Core of Data; REIS 
Program Data. 



The following four exhibits show the patterns of missingness in the data broken down by subject 
and REIS status. Missingness was examined across all six years of data. These tables illustrate 
several findings: 

• Data were available for a greater percentage of total REIS than non-REIS rural districts 
(in both mathematics and reading). 

• Imputing the missing covariates for the REIS districts with achievement data would not 
have added a substantial number of data points to the analysis. 

• The number of districts without covariate data is larger for the non-REIS rural districts 
than the REIS districts. 

• Eor mathematics, data were available for approximately 68 percent of REIS districts and 
55 percent of non-REIS districts (see Exhibits B-4 and B-5). 

• Eor reading, data were available for approximately 65 percent of REIS districts and 53 
percent of non-REIS districts (see Exhibits B-6 and B-7). 
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Exhibit B-4 

Patterns of Missing Data for RLIS Districts 
With and Without Proficiency Data in Mathematics, 2002-08 



Missing Data Patterns 


Math z-score 


Percent 

LEP 


Percent 

Minority 


Percent 

Poverty 


Percent 

Special 

Ed. 


Average 

Total 

Students 


Frequency 


Percent of 
Total 


With Mathematics Score 


X 


X 


X 


X 


X 


X 


5,317 


67.5% 


X 


X 


X 




X 


X 


7 


0.09% 


X 


X 


X 






X 


1 


0.01% 


X 




X 


X 


X 


X 


43 


0.55% 




5,368 


68.15% 




Without Mathematics Score 




X 


X 


X 


X 


X 


2,462 


31.26% 




X 


X 


X 




X 


1 


0.01% 




X 


X 




X 


X 


22 


0.28% 




X 


X 






X 


2 


0.03% 






X 


X 


X 


X 


9 


0.11% 






X 


X 




X 


5 


0.06% 










X 


X 


2 


0.03% 




2,503 


31.78% 



Exhibit reads: For RLIS districts with proficiency data in mathematics, data were 
available on all variables (Math z-score, Percent Minority, Percent LEP, Percent Special 
Education, and total students) for 5,317 districts, about 68 percent of total districts in the 
dataset. 

Source: EDEN-EDEact^; National Eongitudinal School-Eevel State Assessment Score 
Database (NESESASD); Common Core of Data; REIS Program Data. 






Exhibit B-5 

Patterns of Missing Data for Non-RLIS Rurai Districts 
With and Without Proficiency Data in Mathematics, 2002-08 



Missing Data Patterns 


Math z- 
score 


Percent 

LEP 


Percent 

Minority 


Percent 

Poverty 


Percent 

Special 

Ed. 


Average 

Total 

Students 


Erequency 


Percent of 
Total 


With Mathematics Score 


X 


X 


X 


X 


X 


X 


27,810 


54.51% 


X 


X 


X 


X 




X 


1 


<.01% 


X 


X 


X 




X 


X 


915 


1.79% 


X 


X 


X 






X 


46 


0.09% 


X 




X 


X 


X 


X 


303 


0.59% 


X 




X 


X 




X 


3 


0.01% 


X 




X 




X 


X 


78 


0.15% 


X 




X 






X 


5 


0.01% 


X 










X 


3 


0.01% 




29,164 


57.16% 




Without Mathematics Score 




X 


X 


X 


X 


X 


17,686 


34.66% 




X 


X 


X 




X 


13 


0.03% 




X 


X 




X 


X 


1,740 


3.41% 




X 


X 






X 


224 


0.44% 






X 


X 


X 


X 


64 


0.13% 






X 


X 




X 


4 


0.01% 






X 




X 


X 


290 


0.57% 






X 






X 


292 


0.57% 












X 


1,039 


2.04% 


0 


0 


0 


0 


0 


0 


506 


0.99% 














21,858 


42.85% 



Exhibit reads: For non-RLIS rural districts with proficiency data in mathematics, data 
were available on all variables (Math z-score, Percent Minority, Percent LEP, Percent 
Special Education, and total students) for 27,810 districts, about 55 percent of total 
districts in the dataset. 



Source: EDEN-EDEact^; National Eongitudinal School-Eevel State Assessment Score 
Database (NESESASD); Common Core of Data; REIS Program Data. 






Exhibit B-6 

Patterns of Missing Data for RLIS Districts 
With and Without Proficiency Data in Reading, 2002-08 



1 


Missing Data Patterns 


Read z- 
score 


Percent 

LEP 


Percent 

Minority 


Percent 

Poverty 


Percent 

Special 

Ed. 


Average 

Total 

Stndents 


Freqnency 


Percent 
of Total 


With Reading Score 


X 


X 


X 


X 


X 


X 


5,079 


64.48% 


X 


X 


X 




X 


X 


7 


0.09% 


X 


X 


X 






X 


1 


0.01% 


X 




X 


X 


X 


X 


43 


0.55% 




5,130 


65.13% 



Without Reading Score 





X 


X 


X 


X 


X 


2,700 


34.28% 




X 


X 


X 




X 


1 


0.01% 




X 


X 




X 


X 


22 


0.28% 




X 


X 






X 


2 


0.03% 






X 


X 


X 


X 


9 


0.11% 






X 


X 




X 


5 


0.06% 










X 


X 


2 


0.03% 


0 


0 


0 


0 


0 


0 


6 


0.08% 



1,1 M \ 34 . 88 % 



Exhibit reads: For RLIS districts with proficiency data in reading, data were available on 
all variables (Read z-score, Percent Minority, Percent LEP, Percent Special Education, and 
total students) for 5,079 districts, about 65 percent of total districts in the dataset. 

Source: EDEN-EDEact^; National Eongitudinal School-Level State Assessment Score 
Database (NLSLSASD); Common Core of Data; RLIS Program Data. 






Exhibit B-7 

Patterns of Missing Data for Non-RLIS Rurai Districts 
With and Without Proficiency Data in Reading, 2002-08 





Missing Data Patterns 


Read z- 
score 


Percent 

LEP 


Percent 

Minority 


Percent 

Poverty 


Percent 

Special 

Ed 


Average 

Total 

Students 


Erequency 


Percent 
of Total 


With Reading Score 


X 


X 


X 


X 


X 


X 


27,263 


53.43% 


X 


X 


X 


X 




X 


1 


0.00% 


X 


X 


X 




X 


X 


905 


1.77% 


X 


X 


X 






X 


44 


0.09% 


X 




X 


X 


X 


X 


303 


0.59% 


X 




X 


X 




X 


3 


0.01% 


X 




X 




X 


X 


78 


0.15% 


X 




X 






X 


5 


0.01% 


X 










X 


3 


0.01% 




28,605 


56.06% 




Without Reaa 


ing Score 




X 


X 


X 


X 


X 


18,233 


35.74% 




X 


X 


X 




X 


13 


0.03% 




X 


X 




X 


X 


1,750 


3.43% 




X 


X 






X 


226 


0.44% 






X 


X 


X 


X 


64 


0.13% 






X 


X 




X 


4 


0.01% 






X 




X 


X 


290 


0.57% 






X 






X 


292 


0.57% 












X 


1,039 


2.04% 


0 


0 


0 


0 


0 


0 


506 


0.99% 




22,417 


43.95% 



Exhibit reads: For non-RLIS Rural districts with proficiency data in reading, data were 
available on all variables (Math z-score, Percent Minority, Percent LEP, Percent Special 
Education, and total students) for 27,263 districts, about 53 percent of total districts in the 
dataset. 



Source: EDEN-EDEact^; National Eongitudinal School-Eevel State Assessment Score Database 
(NESESASD); Common Core of Data; REIS Program Data. 






Sample Sizes. Below is a comparison of the number of districts included (and not included) in the 
analyses. 

All longitudinal models included only districts with a 6, 7, or 8 locale code. Exhibit B-8 below 
shows the breakdown of districts in the three following categories: 

• REIS districts with a 6, 7, or 8 locale code. 

• Non REIS districts with a 6, 7, or 8 locale code. 



Exhibit B-8 

Overaii Sampie Sizes, NCES Codes 6, 7, and 8 



School Year 


RLIS 6, 7, 8 


Non RLIS 6, 7, 8 


2002-03 


1,729 


8,747 


2003-04 


1,299 


9,186 


2004-05 


1,127 


8,250 


2005-06 


1,188 


8,506 


2006-07 


1,287 


8,461 


2007-08 


1,247 


7,872 


Total 


7,877 


51,022 



Exhibit reads: In 2002-03, there were 1,729 districts that received RLIS funds, and 
8,747 rural districts that did not receive RLIS funds. 

Note: The sample sizes for NCES Codes 6, 7, and 8 reflect the number of districts that 
were eligible for RLIS funds in each school year. 

Source: EDEN-EDEact^; National Longitudinal School-Level State Assessment Score 
Database (NLSLSASD); Common Core of Data. 
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Calculating Proficiency Ratings. Achievement data for the first three years of the longitudinal 
analysis came from National Longitudinal School-Level Assessment Score Database 
(NLSLSASD) and for the last three years from EDEN/EDFact5. The NESESASD proficiency 
percentages were available separately for each grade. Percent proficient per district was therefore 
calculated by dividing the total number of proficient students (for all grades) by the total number 
of students (for all grades): 

Percent Proficient = Total-number-proficient/Total-number- students 

Computing z- scores. Data on percent proficient for each district were used as a measure of 
academic achievement. These data were transformed into z-scores for each year over the five- 
year period of 2002-03 through 2007-08. The values for the z-scores were computed by taking 
the proficiency rating for each school district and subtracting the overall average proficiency 
rating of all districts in that state and then dividing that difference by the standard deviation of 
proficiency ratings for all districts in that state. 

z-scoreij = (District Proficiencyij- Average State Proficiency oj) / State Proficiency std devoj 

Interpreting Impact Estimates. Because data were analyzed using z-scores, changes in the 
estimates represent changes in units of a standard deviation. Eor example, a difference of 0.5 in 
average academic achievement between REIS and non-REIS districts represents a difference of 
one -half of one standard unit (or standard deviation). 

The process of understanding an estimate of the rate of change in academic achievement is 
similar. Eor example, an estimate of 0.5 indicates that with every unit increase in the predictor 
there will be a one-half standard unit change in academic achievement. 

Calculating Covariates. Data were not available for each year on all covariates; therefore, 
average values were calculated using the available data across all years. Averages were 
calculated by summing over the data numerators and denominators. Eor example, percent Special 
Education was calculated by taking the following steps: 

1. Taking the sum of the number of special education students in each district for each 
year that data were available to create the numerator; 

2. Taking the sum of the number of total students (using grade data) in each district for 
each year that data were available to create the denominator; 

3. Removing cases in which the numerator or the denominator had a missing value 
(however, values of zero in the numerator were left in); and 

4. Dividing the numerator by the denominator to get a percent value. 

These same steps were taken for all covariates. One additional step was taken to calculate the 
percent minority. Because data were provided in counts of students by race and ethnicity, a sum 
of all nonwhite students was first calculated. These values (for each district) were then used to 
calculate the overall (numerator) for percent minority. 
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Testing Covariates. In order to ascertain whether or not using the average values for covariates 
would be suitable for modeling, we tested if there were significant differences in the actual 
values for each year that data were available. 

Each covariate was modeled separately taking the form of two-level models with time nested in 
districts. 

Level 1: 

Covij = /3 qj + Pyj {time) + r^j 

Level 2: 

^Oj = ^00 +“ 0 ; 

A j = ^10 

The results indicated that although there were significant differences between the adjusted means 
at the different times points (for some of the covariates) these differences were not functionally 
meaningful. For example, the percent minority was different among the years; however, those 
differences are all in the thousands of a percent. Total students also differed significantly among 
the time points of measurement; however, the significant differences ranged from 5-8 students. 

Assessing Dosage: Dosage was treated as a categorical variable allowing us to test for the 
differences between receiving one to six years of REIS funding and not receiving any funding at 
all. 

Data Used in the Analysis. Data were included in the analysis for all years available during the 
years 2002-03 through 2007-08 for all districts with a locale code of 6, 7, or 8. 

Longitudinal Analyses 

The longitudinal models compared trends in student achievement in REIS districts that those in 
non-REIS districts controlling for several covariates. There were a number of models constructed 
and compared, thereby increasing confidence in the estimates provided. 

For the longitudinal analysis that did not include the dosage variable, we compared models based 
on three different methods of assignment of the REIS indicator variable. Inspection of REIS 
program data indicated that REIS status changed over the years. In case there was an issue with 
the quality of this important piece of the data, several models were constructed which assigned 
the REIS indicator variables in one of the following ways: 

• Always-REIS: Assigned REIS status only if a district received REIS funding in every 
year. 

• Ever- REIS: Assigned REIS status if a district received in at least one year. 

• Changing-REIS: Assigned REIS status differently for each year, solely based on the 
program office eligibility data. 
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All models were two-level with a random component at level 1 and at level 2 for the intercepts 
and time (slope) 

Final Longitudinal Models. The final longitudinal models for both reading and mathematics 
were two-level models with time nested in districts. The RLIS indicator variable was included as 
a level one variable allowing RLIS status to change with each time point (according to the data). 

Level 1 

yij ~ Pqj + Pij (dmcij ) + P 2 j (RLIS _ status ^ ) -l- P^j (dmeij )(RLIS _ status ) 4- 



Level 2 

J^Oj = ^00 + ^01 ) + ^02 ) + 7Tq2 (SpEdj ) + (Total _ stj ) + ttq^ (LEPj ) + uqj 

P\ j = ^10 + j 
Pi j = ^20 
Pi j = ^30 

The following set of tables (Exhibits B-9 through B-14) show the comparison of using different 
RLIS indicator variables in modeling mathematics and reading achievement over time. As can be 
seen in these tables, although the coefficients change slightly, the significance of the treatment 
gains (change over time) is nearly identical in all cases. 

Using the model that contains the variable that allows RLIS status to change over time, the 
following findings emerge: 

• In mathematics, the rate of change for RLIS districts is slightly greater (0.03) than in non- 
RLIS districts (t=4.50, p<.0001). See Exhibit B-9. 

• In reading, the rate of change for REIS districts is slightly greater (0.032) than in non- 
REIS districts (t=3.31; p=.0009). See Exhibit B-12. 



Examination of the variance estimates for three-level models indicated that there was not a significant amount of 
residual variance among the states and, as such, models were constructed as two-level. 
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Exhibit B-9 

Resuits of Modeiing Mathematics Achievement 
Using the Ever-RUS Indicator Variabie 



Effect 


Estimate 


Std Error 


DF 


t Value 


Pr > Itl 


Intercept (non-rlis) 


0.08 


0.01 


8895 


5.81 


<.0001 


Difference at Time 0 


-0.15 


0.03 


8895 


-5.16 


<.0001 



Slope over time 



Non-RLIS 


-0.01 


0.00 


24000 


-3.00 


0.0027 


Trt effect (rlis) 


0.03 


0.01 


24000 


4.97 


<.0001 



Covariates 



Poverty 


-1.06 


0.17 


8895 


-6.05 


<.0001 


Minority 


-0.90 


0.05 


8895 


-18.95 


<.0001 


Spec. Ed. 


-2.41 


0.16 


8895 


-15.40 


<.0001 


Total Students 


0.00004 


0.000004 


8895 


9.38 


<.0001 


Limited English Prof. 


-0.52 


0.11 


8895 


-4.59 


<.0001 



Exhibit reads: The estimate for the intercept in 2002-03 for non-RLIS districts is 0.08, with 
a standard error of 0.01. The degrees of freedom for this estimate are 8,895 and the t-value is 
5.81 with an associated prohahility of p<.0001. 

Source: EDEN-EDEact^; National Eongitudinal School-Eevel State Assessment Score 
Database (NESESASD); Common Core of Data; REIS Program Data. 






Exhibit B-10 

Resuits of Modeiing Mathematics Achievement 
Using the Always-RLIS Indicator Variabie 



Effects 


Estimate 


Std Error 


DF 


t Value 


Pr > Itl 


Intercept (non-rlis) 


0.07 


0.01 


7697 


4.90 


<.0001 


Difference at Time 0 


-0.04 


0.05 


7697 


-0.93 


0.3532 


Slope over time 


Non-RLIS 


-0.01 


0.00 


21000 


-2.95 


0.0031 


Trt effect (rlis) 


0.03 


0.01 


21000 


3.87 


0.0001 



Covariates 



Poverty 


-1.45 


0.21 


7697 


-7.06 


<.0001 


Minority 


-0.92 


0.05 


7697 


-17.48 


<.0001 


Spec. Ed. 


-2.42 


0.17 


7697 


-14.57 


<.0001 


Total Students 


0.00003 


0.000004 


7697 


7.78 


<.0001 


Limited English Prof. 


-0.48 


0.12 


7697 


-3.89 


0.0001 



Exhibit reads: The estimate for the intercept in 2002-03 for non-RLIS districts is 0.07, 
with a standard error of 0.01. The degrees of freedom for this estimate are 7,697 and the 
t- value is 4.90 with an associated prohahility of p<.0001. 

Source: EDEN-EDEact^; National Eongitudinal School-Eevel State Assessment Score 
Database (NESESASD); Common Core of Data; REIS Program Data. 






Exhibit B-11 

Resuits of Modeiing Mathematics Achievement 
Using the Changing-RUS Indicator Variabie 



Effect 


Estimate 


Std Error 


DE 


t Value 


Pr > Itl 


Intercept (non-rlis) 


0.05 


0.01 


8896 


4.49 


<.0001 


Difference at Time 0 


-0.07 


0.02 


787 


-3.27 


0.0011 


Slope over time 


Non-REIS 


-0.01 


0.003 


24000 


-2.33 


0.0196 


Trt effect (rlis) 


0.03 


0.01 


24000 


4.50 


<.0001 



Covariates 



Poverty 


-1.35 


0.16 


8896 


-8.51 


<.0001 


Minority 


-0.90 


0.05 


8896 


-18.89 


<.0001 


Spec. Ed. 


-2.40 


0.16 


8896 


-15.38 


<.0001 


Total Students 


0.00003 


0.000004 


8896 


9.07 


<.0001 


Eimited English Prof. 


-0.48 


0.11 


8896 


-4.24 


<.0001 



Exhibit reads: The estimate for the intercept in 2002-03 for non-RLIS districts is 0.05, 
with a standard error of 0.01. The degrees of freedom for this estimate are 8,896 and the 
t- value is 4.49 with an associated prohahility of p<.0001. 

Source: EDEN-EDEact^; National Eongitudinal School-Eevel State Assessment Score 
Database (NESESASD); Common Core of Data; REIS Program Data. 






Exhibit B-12 

Resuits of Modeiing Reading Achievement 
Using the Ever-RUS Indicator Variabie 



Effect 


Estimate 


Std Error 


DF 


t Value 


Pr > Itl 


Intercept (non-rlis) 


0.06 


0.01 


8892 


4.55 


<.0001 


Difference at Time 0 


-0.09 


0.03 


8892 


-3.17 


0.0015 


Slope over time 


Non-RLIS 


-0.01 


0.003 


23000 


-2.63 


0.0084 


Trt effect (rlis) 


0.02 


0.01 


23000 


2.77 


0.0056 



Covariates 



Poverty 


-1.28 


0.17 


8892 


-7.59 


<.0001 


Minority 


-0.92 


0.05 


8892 


-20.10 


<.0001 


Spec. Ed. 


-2.70 


0.15 


8892 


-17.89 


<.0001 


Total Students 


0.00003 


0.000004 


8892 


8.68 


<.0001 


Limited English Prof. 


-1.08 


0.11 


8892 


-9.84 


<.0001 



Exhibit reads: The estimate for the intercept in 2002-03 for non-RLIS districts is 0.06, 
with a standard error of 0.01. The degrees of freedom for this estimate are 8,892 and the 
t- value is 4.55 with an associated prohahility of p<.0001. 

Source: EDEN-EDEact^; National Eongitudinal School-Eevel State Assessment Score Database 
(NESESASD); Common Core of Data; REIS Program Data. 






Exhibit B-13 

Resuits of Modeiing Reading Achievement 
Using the Always-RLIS Indicator Variabie 



Effect 


Estimate 


Std Error 


DF 


t Value 


Pr > Itl 


Intercept (non-rlis) 


0.05 


0.01 


7695 


3.52 


0.0004 


Difference at Time 0 


0.05 


0.05 


7695 


1.12 


0.2616 


Slope over time 


Non-RLIS 


-0.01 


0.00 


20000 


-2.62 


0.0089 


Trt effect (rlis) 


0.02 


0.01 


20000 


2.10 


0.0353 



Covariates 



Poverty 


-1.80 


0.20 


7695 


-9.07 


<.0001 


Minority 


-0.94 


0.05 


7695 


-18.49 


<.0001 


Spec. Ed. 


-2.73 


0.16 


7695 


-17.09 


<.0001 


Total Students 


0.00003 


0.000004 


7695 


7.19 


<.0001 


Limited English Prof. 


-1.01 


0.12 


7695 


-8.59 


<.0001 



Exhibit reads: The estimate for the intercept in 2002-03 for non-RLIS districts is 0.05, with a 
standard error of 0.01. The degrees of freedom for this estimate are 7,695 and the t-value is 
3.52 with an associated prohahility of p=.0004. 

Source: EDEN-EDEact^; National Eongitudinal School-Eevel State Assessment Score 
Database (NESESASD); Common Core of Data; REIS Program Data. 






Exhibit B-14 

Resuits of Modeiing Reading Achievement 
Using the Changing-RUS Indicator Variabie 



Effect 


Estimate 


Std Error 


DF 


t Value 


Pr > Itl 


Intercept (non-rlis) 


0.05 


0.01 


8893 


3.95 


<.0001 


Difference at Time 0 


-0.05 


0.02 


756 


-2.18 


0.0299 


Slope over time 


Non-RLIS 


-0.01 


0.00 


23000 


-2.63 


0.0086 


Trt effect (rlis) 


0.02 


0.01 


23000 


3.31 


0.0009 



Covariates 



Poverty 


-1.50 


0.16 


8893 


-9.63 


<.0001 


Minority 


-0.92 


0.05 


8893 


-20.06 


<.0001 


Spec. Ed. 


-2.70 


0.15 


8893 


-17.88 


<.0001 


Total Students 


0.00003 


0.000004 


8893 


8.42 


<.0001 


Limited English Prof. 


-1.05 


0.11 


8893 


-9.60 


<.0001 



Exhibit reads: The estimate for the intercept in 2002-03 for non-RLIS districts is 0.05, 
with a standard error of 0.01. The degrees of freedom for this estimate are 8,893 and the 
t- value is 3.95 with an associated prohahility of p<.0001. 

Source: EDEN-EDEact^; National Eongitudinal School-Eevel State Assessment Score Database 
(NESESASD); Common Core of Data; REIS Program Data. 






Exhibit B-15 shows the average proficiency rating for the REIS and non-REIS (rural and not 
rural) districts for each year from 2002-03 through 2007-08. 



Exhibit B-15 

Average Mathematics and Reading Proficiency by Group, 2002-08 





RLIS, 678 


Non RLIS, 678 


School 


Variable 


N 


N 


Mean 


Std 


N 


N 


Mean 


Std 


Year 


Ohs 


(%) 


Dev 


Ohs 


(%) 


Dev 


2002- 


Ave. Prof. 
Math 


1729 


838 


0.50 


0.20 


8747 


3645 


0.57 


0.21 


03 


Ave. Prof. 
Read 


765 


0.64 


0.17 


3303 


0.69 


0.14 


2003- 


Ave. Prof. 
Math 


1299 


578 


0.47 


0.22 


9186 


3831 


0.59 


0.21 


04 


Ave. Prof. 
Read 


496 


0.63 


0.19 


3648 


0.71 


0.15 


2004- 


Ave. Prof. 
Math 


1127 


643 


0.48 


0.23 


8250 


3317 


0.62 


0.21 


OS 


Ave. Prof. 
Read 


559 


0.65 


0.19 


3259 


0.74 


0.15 


2005- 


Ave. Prof. 
Math 


1188 


912 


0.61 


0.17 


8506 


5100 


0.65 


0.19 


06 


Ave. Prof. 
Read 


912 


0.67 


0.16 


5100 


0.71 


0.18 


2006- 


Ave. Prof. 
Math 


1287 


1238 


0.64 


0.16 


8461 


6920 


0.69 


0.18 


07 


Ave. Prof. 
Read 


1237 


0.68 


0.16 


6905 


0.72 


0.17 


2007- 


Ave. Prof. 
Math 


1247 


1160 


0.63 


0.18 


7872 


6376 


0.70 


0.18 


OS 


Ave. Prof. 
Read 


1160 


0.67 


0.19 


6367 


0.75 


0.17 



Exhibit reads: For the 2002-03 school year, data were available for 1,729 RLIS districts. 
There were data for 838 districts in Mathematics proficiency, and the average Mathematics 
proficiency for these RLIS districts was .50 (with a standard deviation of .20). There were 
data for 765 districts in Reading proficiency, and the average Reading proficiency for these 
RLIS districts was .64 (with a standard deviation of .17). 

Source: EDEN-EDEact^; National Eongitudinal School-Eevel State Assessment Score 
Database (NESESASD); Common Core of Data. 
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Analysis of Districts with Six Years of Data 



Because there was a substantial amount of missing data and the source for the outcome measure 
changed midpoint during the data collection period, there was some concern that those districts 
submitting data to the two systems could be systematically different thereby skewing the final 
results. Therefore, we conducted an additional analysis looking only at those districts that have 
six years of data. The models for the mathematics and reading achievement were the same as 
those used for the main analysis. Time was nested within district and the RLIS indicator variable 
was treated as time variant. The final data sets included approximately 20 percent of the districts 
in the full data set. 

Results are similar to those from the models that use the full data sets. 

• In mathematics, the rate of change for RLIS districts is slightly greater (0.02) than in non- 
RLIS districts (t=2.80, p=.038). See Exhibit B-16. 

• In reading, the rate of change for RLIS districts is slightly greater (0.03) than in non- 
RLIS districts (t=2.60; p=.009). See Exhibit B-17. 



Exhibit B-16 

Resuits of Modeiing Mathematics Achievement 
Using Oniy Those Districts With Six Years of Data 



Effect 


Estimate 


Std Error 


DF 


t Value 


Pr > Itl 


Intercept (non-rlis) 


0.04 


0.02 


2033 


2.04 


0.0413 


Difference at Time 0 


-0.07 


0.03 


275 


-2.10 


0.0369 


Slope over time 


Non-RLIS 


-0.01 


0.00 


10000 


-1.66 


0.0964 


Trt effect (rlis) 


0.02 


0.01 


10000 


2.08 


0.0378 


Covariates 


Poverty 


-1.47 


0.28 


2033 


-5.24 


<.0001 


Minority 


-0.33 


0.10 


2033 


-3.46 


0.0006 


Spec. Ed. 


-2.40 


0.38 


2033 


-6.35 


<.0001 


Total Students 


0.00005 


0.000016 


2033 


3.25 


0.0012 


Limited English Prof. 


-1.89 


0.25 


2033 


-7.60 


<.0001 



Exhibit reads: The estimate for the intercept in 2002-03 for non-RLIS districts is 0.04, 
with a standard error of 0.02. The degrees of freedom for this estimate are 2,033 and the 
t-value is 2.04 with an associated probability of p=.0413. 

Source: EDEN-EDEact^; National Longitudinal School-Level State Assessment Score Database 
(NLSLSASD); Common Core of Data; RLIS Program Data. 
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Exhibit B-17 

Resuits of Modeiing Reading Achievement 
Using Oniy Those Districts With Six Years of Data 



Effect 


Estimate 


Std Error 


DF 


t Value 


Pr > Itl 


Intercept (non-rlis) 


0.05 


0.02 


1647 


2.54 


0.0112 


Difference at Time 0 


-0.10 


0.04 


221 


-2.71 


0.0072 


Slope over time 


Non-RLIS 


-0.01 


0.00 


8262 


-2.10 


0.0356 


Trt effect (rlis) 


0.03 


0.01 


8262 


2.60 


0.0092 


Covariates 


Poverty 


0.27 


0.31 


1647 


0.86 


0.3878 


Minority 


-0.80 


0.10 


1647 


-8.18 


<.0001 


Spec. Ed. 


-4.98 


0.43 


1647 


-11.66 


<.0001 


Total Students 


0.00001 


0.000016 


1647 


0.57 


0.5715 


Limited English Prof. 


-2.41 


0.23 


1647 


-10.38 


<.0001 



Exhibit reads: The estimate for the intercept in 2002-03 for non-RLIS districts is 0.05, 
with a standard error of 0.02. The degrees of freedom for this estimate are 1,647 and the 
t- value is 2.54 with an associated prohahility of p=.0112. 

Source: EDEN-EDEact^; National Longitudinal School-Level State Assessment Score Database 
(NLSLSASD); Common Core of Data; REIS Program Data. 






Dosage Analysis 

In order to examine the association between various years of receiving RLIS funds and student 
achievement we chose to model dosage as a categorical variable. This treatment of the variable 
allowed us to compare receiving one to six years of funding to not receiving any funding at all. 
Because the results suggest that receiving the funding for five or six years (depending on the 
subject tested) was related to increased student achievement we did not look at additional 
comparisons such as receiving the funding for one versus two years or three years. 

Final Dosage Models: The data were fit as two-level longitudinal models with level 1 
representing time and level 2 representing districts. Covariates and dosage levels were entered 
into the model as level 2 variables. The models fit for both math and reading were as follows: 

Level 1 

yij = ^0 j + P\j {timeij ) + Ty- 



Level 2 

Poj = ^00 + ^QiiPoVj ) + 7TQ2 (Mitij ) + ;to 3 (SpEdj ) + 7Tq 4 {Total _ st j ) + {LEPj ) 

+ X {Dosage j ) + UQj 
P\j = ^10 + X^2p {Dosage j ) + u^j 
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Exhibit B-20 shows the average proficiencies in mathematics and reading in terms of average percent proficient, standard deviation 
and sample sizes for each year and each level of REIS dosage. The data is disaggregated into dosage groups: districts that did not 
receive REIS funding and districts that received one to six years of REIS funding between 2002 and 2008. 



Exhibit B-18 

Average Proficiency for Mathematics and Reading 
for Non-RLIS and RLIS Districts by Year and Dosage 







Non-RLIS 


One Year 


2 years 


School 




N 




Mean 


Std 


N 




Mean 


Std 


N 




Mean 


Std 


Year 


Variable 


Obs 


N 


(%) 


Dev 


Obs 


N 


(%) 


Dev 


Obs 


N 


(%) 


Dev 


2002-03 


Ave. Prof. 
Math 


8330 


3447 


0.57 


0.21 


601 


272 


0.54 


0.18 


264 


125 


0.50 


0.22 


Ave. Prof. 
Read 


3130 


0.70 


0.14 


255 


0.67 


0.13 


110 


0.64 


0.18 


2003-04 


Ave. Prof. 
Math 


8371 


3438 


0.60 


0.21 


570 


254 


0.58 


0.19 


263 


113 


0.50 


0.23 


Ave. Prof. 
Read 


3268 


0.71 


0.15 


239 


0.69 


0.14 


97 


0.66 


0.19 


2004-05 


Ave. Prof. 
Math 


7425 


2847 


0.63 


0.21 


521 


269 


0.63 


0.18 


161 


88 


0.52 


0.23 


Ave. Prof. 
Read 


2832 


0.75 


0.14 


254 


0.73 


0.13 


75 


0.68 


0.16 


2005-06 


Ave. Prof. 
Math 


7725 


4590 


0.65 


0.19 


525 


339 


0.63 


0.16 


166 


116 


0.61 


0.17 


Ave. Prof. 
Read 


4590 


0.72 


0.18 


339 


0.69 


0.17 


116 


0.68 


0.15 


2006-07 


Ave. Prof. 
Math 


7773 


6251 


0.69 


0.18 


527 


502 


0.68 


0.15 


171 


164 


0.66 


0.16 


Ave. Prof. 
Read 


6266 


0.73 


0.17 


502 


0.70 


0.15 


164 


0.69 


0.14 


2007-08 


Ave. Prof. 
Math 


7159 


5822 


0.70 


0.19 


520 


380 


0.68 


0.16 


167 


140 


0.65 


0.17 


Ave. Prof. 
Read 


5831 


0.75 


0.17 


380 


0.72 


0.16 


140 


0.69 


0.16 



(Continues on next page) 
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Exhibit B-18 (continued) 

Average Proficiency for Mathematics and Reading 
for Non-RLIS and RLIS Districts by Year and Dosage 



School 

Year 


Variable 


3 years 


4 years 


5 years 


6 years 


N 

Ohs 


N 


Mean 

(%) 


Std 

Dev 


N 

Ohs 


N 


Mean 

(%) 


Std 

Dev 


N 

Ohs 


N 


Mean 

(%) 


Std 

Dev 


N 

Ohs 


N 


Mean 

(%) 


Std 

Dev 


2002-03 


Ave. Prof. 
Math 


122 


56 


0.44 


0.21 


186 


80 


0.48 


0.22 


233 


148 


0.45 


0.23 


740 


355 


0.48 


0.22 


Ave. Prof. 
Read 


50 


0.59 


0.20 


71 


0.64 


0.17 


137 


0.62 


0.17 


315 


0.63 


0.19 


2003-04 


Ave. Prof. 
Math 


121 


54 


0.49 


0.23 


187 


89 


0.47 


0.21 


233 


127 


0.45 


0.22 


740 


334 


0.48 


0.22 


Ave. Prof. 
Read 


48 


0.61 


0.21 


81 


0.60 


0.20 


116 


0.63 


0.18 


295 


0.63 


0.19 


2004-05 


Ave. Prof. 
Math 


118 


73 


0.52 


0.22 


183 


96 


0.51 


0.22 


229 


145 


0.50 


0.22 


740 


442 


0.48 


0.24 


Ave. Prof. 
Read 


67 


0.62 


0.21 


83 


0.65 


0.20 


135 


0.64 


0.18 


372 


0.65 


0.19 


2005-06 


Ave. Prof 
Math 


120 


83 


0.59 


0.17 


185 


141 


0.58 


0.19 


233 


174 


0.64 


0.16 


740 


569 


0.60 


0.17 


Ave. Prof 
Read 


83 


0.66 


0.17 


141 


0.65 


0.18 


174 


0.72 


0.16 


569 


0.66 


0.16 


2006-07 


Ave. Prof 
Math 


122 


110 


0.63 


0.17 


182 


173 


0.63 


0.17 


233 


229 


0.65 


0.17 


740 


713 


0.63 


0.16 


Ave. Prof 
Read 


110 


0.66 


0.17 


173 


0.68 


0.16 


229 


0.71 


0.16 


714 


0.67 


0.16 


2007-08 


Ave. Prof 
Math 


120 


100 


0.62 


0.18 


182 


160 


0.63 


0.17 


231 


220 


0.67 


0.17 


740 


705 


0.62 


0.18 


Ave. Prof 
Read 


100 


0.64 


0.20 


160 


0.68 


0.18 


220 


0.72 


0.16 


705 


0.65 


0.20 


Exhibit reads: Eor the 2002-03 school 


1 year, d 


ata were availab 


le for 8,330 non-RElS Rural districts. There were data ] 


for 3,477 districts in 



Mathematics proficiency. The average Mathematics proficiency for these districts was .57 (with a standard deviation of .21). There were data for 
3,130 districts in Reading proficiency. The average Reading proficiency for these RLIS districts was .70 (with a standard deviation of .14). 



Source: EDEN-EDTact^; National Eongitudinal School-Eevel State Assessment Score Database (NESESASD); Common Core of Data. 






Characteristics of RLIS Implementation and Student Achievement 

Given the nested structure of the data, a two-level hierarchical linear modeling approach was 
employed in these analyses. Each model was conceived as having two levels with district (level 
1) nested within state (level 2). The variables of interest (goals, activities, and ratings of state 
technical assistance) were included as level 1 covariates along with the following controls: 
mathematics achievement in 2002-03, reading achievement in 2002-03, district size, years 
receiving RLIS funds, and locale code. 

The model examining the relationship between identifying an RLIS goal as an area of focus and 
student achievement was conceptualized as: 

Level 1 

-I- (mathprcij ) + (readingprC/j ) -t- P^j {districtsize) + P^- (RLISyears) -f 
P^j(RLISgoals) + P^j (Locale) + r.j 

Level 2 

/^oj = ^00 + ^01 (region) + u,,. 

A j ~ ^10 

Pi j ~ ^20 

/?3 J = 71 
Pa j ~ ^40 

Psj = ^50 + ^sfregion) + u^. 

Pb j ~ ^60 

The model examining the relationship between implementing particular RLIS activities and 
student achievement was conceptualized as follows. 



Level 1 

y^j ~ Poj + Pij (mathpre-j ) + P^j (readingpre ^ ) -i- P^j (districtsize) -i- P^j (RLISyears) + 
P ^j(RLlSactiviies) + P^j (Locale) -t- r.j 
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Level 2 




^0. 


o 

o 

II 


+ ^Qi(region) + UQj 


A. 


~ ^10 




A. 


II 




A. 


~ ^30 




A. 


c 

II 




A. 


II 

U\ 

O 


+ 7V^^{region) + u^j 


A. 


$ 

II 
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APPENDIX C: STATE COORDINATOR INTERVIEW GUIDE 



Evaluation of the Implementation of the 
Rural and Low-Income School (REIS) Program 



Introduction 

1 . How long have you been the RLIS/REAP state coordinator? 

NOTE: IE THE STATE COORDINATOR HAS BEEN IN THE JOB SIX MONTHS OR 
EESS, ASK: Is the previous state coordinator available for us to talk to if there are questions 
that should more appropriately be answered by him or her? 

• (If yes) Could you let us know if we come to any such questions, so we can ask them of the 
previous state coordinator instead? 

• (If no) Could you please go ahead and answer any such questions to the best of your 
ability? We understand if your knowledge of past decisions or activities is incomplete. 

2. Please describe your role and responsibilities in administering the REIS Program. 

3. Have your role and/or responsibilities changed over time? (If so) Please describe. 



Goals and Priorities 



Next, we would like to ask about the goals and priorities of your state’s REIS Program. 

4. What are the current goals and priorities of your state’s REIS Program? 

PROBE: 

• Are the current goals and priorities of your state’s REIS Program intended to address 
specific challenges faced by rural districts in your state? 

5. Have the goals and priorities of your state’s REIS Program changed over time? 

(If yes) What goals changed, and why? 

6. Are future reassessments of your state’s goals planned? (If yes) Who would be involved in 
doing such a reassessment? How would such a reassessment be done? 

7. Do the REIS-funded districts in your state set their own goals and priorities distinct from 
those established by the state program? (If yes) How do the districts communicate these goals 
and priorities to you? 
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Administration 



Next, we would like you to describe how the distribution of RLIS funds is administered. 

8. How do you notify RLIS-eligible districts as to their eligibility for funds? Have your 
communication mechanisms changed over time? 

9. How does the process by which RLIS funds are distributed to eligible districts fit in with 
your overall policies and procedures for School Support and Technology Programs and other 
programs for LEAs? 

10. Does your state require the eligible districts to submit an application for or otherwise make a 
formal request for the RLIS funds? (If yes) Please describe the application/request process 
that must be followed by the districts. 

11. Does the state provide any guidance and assistance for eligible districts for completing the 
application process? 

12. Has your process for the distribution of RLIS funds changed over time? (If yes) How? 



State Monitoring, Technical Assistance, and Evaluation 

Now, we would like to ask you some questions regarding your state’s processes for monitoring 
and evaluating the RLIS Program. 

13. Do you provide districts with information on the application process, the state’s goals and 
priorities, or guidance on the RLIS Program in general? (If yes) Can you share those 
documents with us? 

14. How do you monitor your state’s RLIS Program? 

15. Do you provide any forms of technical assistance to RLIS districts? (If yes) What types of 
assistance do you offer? Can you share any technical assistance-related documents with us? 

16. Have data on the RLIS Program been collected? (If yes) Which types? Can you share any of 
your raw data with us? 

PROBE: 

• How often are data collected and analyzed? 

• How do you use the data you collect? 

• What staff members are responsible for collecting and analyzing data? 

17. Do you conduct evaluations of your state’s RLIS Program? (If yes) How so? Have you 
generated any reports or self-evaluations? (If yes) Can you share them with us? What have 
the reports found? 



Appendix C 



113 






Related State Policies and Initiatives 



18. Are there any policy initiatives or funding sources in your state that support, complement, or 
supplement the RLIS Program? (If yes) Can you describe them for me? 

PROBE: 

• How are these programs coordinated at the state level? 

• Does the state provide guidance and assistance to districts to help them effectively 
coordinate the funds available from these various programs? 

19. Are there any policy initiatives or funding sources in your state that compete with the RLIS 
Program? (If yes) Can you describe them for me? 



Wrap-Up 

20. Now that you understand the types of information we are looking for, is there anything else 
you think we should know about your state’s RLIS Program? Do you have any suggestions 
for improvement of the RLIS Program? 

21. 1 want to confirm that you are going to email/mail me the following documents that we 
discussed in the interview: (list documents, such as program data, evaluation data, reports, 
guidance, etc.) When should I expect those documents? 

22. Do you have any questions about the study? 

Thank you for your time! 
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Evaluation of the Implementation of the Rural and Low-Income School (RLIS) Program 
COVER PAGE FOR STATE COORDINATOR INTERVIEW 



Note: The interview will be scheduled and our study introduced and explained in a set-up call in 
advance of the interview itself. We will, however, want to review some or all of this information 
at the time of the interview. 

Hello. This is [name], from Berkeley Policy Associates. As we discussed (refer to last time we 
spoke) we are conducting a study under contract with the Policy and Program Studies Service of 
the U.S. Department of Education to learn about how the Rural and Low-Income School 
Program is being implemented at the state and local levels. We are particularly interested in 
learning about your goals and priorities, and uses of funds. As you know, you are one of nine 
states that have been selected for an in-depth interview regarding your state’s RLIS Program. 
The information you provide will also inform subsequent data collection activities. 

(Refer to discussion of interview length held during set-up call.) Our interview will take about 
one hour — Is this still a good time for you? (Negotiate new time or divide interview into two 
sessions as needed.) Thank you so much for your time; we know how busy you must be. 

Please be as honest and candid as possible. Any information regarding your successes and 
challenges will help us understand the overall picture, and we especially appreciate learning 
about your experiences — both positive and negative. In our reporting, we will not associate 
responses or findings with individual names or the names of the states. 

We have two interviewers on the line, one asking the questions and the other taking notes, who 
may ask questions as well. We would like to make a recording of the conversation as a back-up 
to our note taking; this recording would be erased as soon as we have verified that our notes are 
complete. Do we have your permission to record our conversation? 

There will be time for you to ask questions about the study at the end of the interview, but if you 
need anything clarified during our discussion, please let us know. Are you ready to begin? 

Note: Prior to the interview, we will review written materials on the particular state, and tailor 
the topic guide slightly as appropriate. 
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APPENDIX D: DISTRICT COORDINATOR INTERVIEW GUIDE 



Evaluation of the Implementation of the 
Rural and Low-Income School (REIS) Program 



Introduction 

1 . What is your job title? 

2. Please describe your role and responsibilities in administering the RLIS program in your 
district. 

3. How long have you been responsible for administering the RLIS program in your district? 
Have your role and/or responsibilities changed over time? (If so) Please describe. 

NOTE: IF IN THIS POSITION SIX MONTHS OR LESS, ASK: Is the person who was 
previously responsible for administering the RLIS program in your district available to talk if 
there are questions that should more appropriately be answered by him or her? 

• (If yes) Could you let us know if we come to any such questions, so we can ask them of the 
previous state coordinator instead? 

• (If no) Could you please go ahead and answer any such questions to the best of your 
ability? We understand if your knowledge of past decisions or activities is incomplete. 



Goals and Priorities 



Next, we would like to ask about the goals and priorities for your district’s RLIS funds that you 

received for the 2007-2008 school year. 

4. What were the goals and priorities for the use of your district’s RLIS funds for the 2007- 
2008 school year? 

5. How did you determine what the goals and priorities for your district should be? Do you 
conduct needs assessments, or other assessments of what your district’s needs are? (If yes) 
How often? How do you use the data you collect to determine what the goals and priorities 
for your district should be? 

6. Do you have any documentation of how you determined what the goals and priorities for 
your district should be, such as needs assessment reports? (If yes) Can you share those 
documents with us? 

7. Have the goals and priorities for the use of your district’s RLIS funds changed over time? 
(If yes) What goals changed, and why? 
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8. Are future reassessments of your district’s goals planned? (If yes) Who would be involved in 
doing such a reassessment? How would such a reassessment be done? 



Uses of Funds 

9. How is your district using RLIS funds to achieve the goals you have set? That is, to what 
specific uses are you putting the RLIS funds? 

10. Do you have any documents you could share on 2007-08 RLIS expenditures? 

1 1. Is your district in school improvement? (If yes) Are you using RLIS funds to achieve specific 
goals for school improvement? 

12. What progress has your district made toward reaching the goals you have set? How do you 
measure that progress? 



State Monitoring and Technical Assistance 

13. Does the state provide any guidance or assistance for applying for RLIS funds? If so, please 
describe the assistance and how it was used. 

14. Does the state provide any guidance or assistance in setting goals related to RLIS and 
identifying effective strategies for reaching those goals? If so, please describe the assistance 
and how it was used. 

15. Have you received any monitoring visits or been the subject of any other monitoring efforts 
from your state that are related to your RLIS program during the 2007-08 or the 2008-09 
school years? (If yes) What did the monitoring consist of, and what were the results as they 
related to the RLIS program in your district? How often did the monitoring occur? 

16. Does the state provide any guidance or assistance in evaluating the RLIS Program at the 
district level, or in measuring one’s progress toward reaching the goals that have been set for 
your district? If so, please describe the assistance and how it was used. 



Wrap-Up 

17. Now that you understand the types of information we are looking for, is there anything else 
you think we should know about your state’s RLIS Program? 

18. Do you have any suggestions for improvement of the RLIS Program? 

19. 1 want to confirm that you are going to email/mail me the following documents that we 

discussed in the interview: (list documents, such as needs assessment reports) When should I 
expect those documents? 

20. Do you have any questions about the study? Thank you for your time! 
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Evaluation of the Implementation of the Rural and Low-Income School (RLIS) Program 
COVER PAGE FOR DISTRICT COORDINATOR INTERVIEW GUIDE 

Note: The interview will be scheduled and our study introduced and explained in a set-up call in 
advance of the interview itself. We will, however, want to review some or all of this information 
at the time of the interview. 

Hello. This is [name], from Berkeley Policy Associates. As we discussed (refer to last time we 
spoke) we are conducting a study under contract with the Policy and Program Studies Service of 
the U.S. Department of Education to learn about how the Rural and Low-Income Schools 
Program is being implemented at the state and local levels. We are particularly interested in 
learning about your goals and priorities and uses of funds. As you know, your district is in one of 
nine states that have been selected for in-depth interviews regarding the RLIS Program. (Refer to 
discussion of interview length held during set-up call.) Our interview will take about a half 
hour — Is this still a good time for you? (Negotiate new time as needed.) Thank you so much for 
your time; we know how busy you must be. 

Please be as honest and candid as possible. Any information regarding your successes and 
challenges will help us understand the overall picture, and we especially appreciate learning 
about your experiences — both positive and negative. In our reporting, we will not associate 
responses or findings with individual names or the names of the districts or states. 

We have two interviewers on the line, one asking the questions and the other taking notes, who 
may ask questions as well. We would like to make a recording of the conversation as a back-up 
to our note taking; this recording would be erased as soon as we have verified that our notes are 
complete. Do we have your permission to record our conversation? 

There will be time for you to ask questions about the study at the end of the interview, but if you 
need anything clarified during our discussion, please let us know. Are you ready to begin? 
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APPENDIX E: REIS STATE COORDINATOR SURVEY 

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of 
information unless such collection displays a valid OMB control number. The valid OMB control number for this 
information collection is xxxx-xxxx. The time required to complete this information collection is estimated to 
average 20 minutes per response, including the time to review instructions, search existing data resources, gather the 
data needed, and complete and review the information collection. If you have any comments concerning the 
accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of 
Education, Washington, D.C. 20202-4651. If you have comments or concerns regarding the status of your individual 
submission of this form, write directly to: Erica Lee, U.S. Department of Education, Office of Planning, Evaluation 
and Policy Development (OPEPD), Policy and Program Studies Service, 400 Maryland Avenue, SW, Washington, 
D.C. 20202. 

Responses to this data collection will be used only for statistical purposes. The reports prepared for this study will 
summarize findings across the sample and will not associate responses with a specific district/state or individual. We 
will not provide information that identifies you or your district/state to anyone outside the study team, except as 
required by law. 

Your cooperation in completing this survey will help to make the results of this evaluation comprehensive, reliable, 
and timely. 



1) Does your state use funds from the state RLIS allocation for administrative purposes? 

No — > Go to question 3 
Yes —>■ Go to question 2 

2) What proportion of the state RLIS allocation was used for administrative purposes 
during the 2007-2008 school year? 

None 

Less than 5 percent 
5 percent 

3) Do all eligible districts in your state receive RLIS funds? 

No 

Yes 

4) What type of protocol must eligible districts in your state follow in order to receive 
their RLIS allocation? 

No formal process or protocol, all districts receive all funds for which they are eligible 

An application for RLIS funds that is part of a comprehensive district improvement plan or other district 
application for funds 

A separate RLIS -specific application form or process 
Other (please specify) 

If you selected other, please specify: 
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5) How do you inform districts that they are eligible to receive RLIS funds? 

(Mark all that apply) 

Email 

Posting on Web site 

Letter 

Phone call 

Announcement at state conference or workshop 
Other (please specify) 

If you selected other, please specify; 

6) In addition to the primary goal of making Adequate Yearly Progress (AYP), to what 
extent are the following a focus for the use of RLIS funds in your state? 





H 


Minimal 

focus 


Moderate 

focus 


Major 

focus 


a. Increasing student achievement in a particular subject area 










b. Reducing the high school dropout rate 










c. Improving the quality of instruction 










d. Improving the ability of English language learners to achieve 
proficiency in English and reach high academic standards 










e. Ensuring that all students will be educated in learning environments 
that are safe, drug free, and conducive to learning 










f. Addressing issues specific to rural location, e.g., retaining teachers, 
providing distance learning opportunities, etc. 











7) In addition to the primary goal of making AYP and those listed above, are RLIS funds 
used to meet any other goals? 

No — > Go to question 9 
Yes ^ Go to question 8 

8) Please list the additional goals. 



9) To what extent are the following activities a focus for the use of RLIS funds in your 
state? 





■ 


Minimal 

focus 


Moderate 

focus 


Major 

focus 


a. Teacher recruitment and retention, including the use of signing 
bonuses and other financial incentives 










b. Teacher professional development 










c. Educational technology, including software and hardware 










d. Parental involvement activities 










e. Activities authorized under the Safe and Drug-Eree Schools Program 










f Activities authorized under Title I, Part A 










g. Language instruction for ELL/LEP students 
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10) Are there any other activities that are a focus for the use of RLIS funds in your state? 

No —>■ Go to question 12 
Yes —y Go to question 1 1 

11) Please list other activities that are a focus for the use of RLIS funds in your state. 



12) To what extent do you feel your state has made progress toward your RLIS goals and 
objectives, including the goal of making Adequate Yearly Progress? 

No progress —y Go to question 16 
Minimal progress —y Go to question 1 3 
Moderate progress —y Go to question 1 3 

Goals and objectives have been accomplished — > Go to question 13 

13) To what extent has the RLIS funding contributed to your state's progress towards its 
RLIS goals and objectives? 





No 

contribution 


Minimal 

contribution 


Moderate 

contribution 


Major 

contribution 


a. Making Adequate Yearly Progress 










b. Increasing student achievement in a particular 
subject area 










c. Reducing the high school dropout rate 










d. Improving the quality of instruction 










e. Improving the ability of English language learners 
to achieve proficiency in English and reach high 
academic standards 










f Ensuring that all students will be educated in 
learning environments that are safe, drug free, and 
conducive to learning 










g. Addressing issues specific to rural location, e.g., 
retaining teachers, providing distance learning 
opportunities, etc. 











14) Has RLIS funding contributed to your state's progress toward any other goals and 
objectives? 

No — > Go to question 16 
Yes Go to question 15 

15) Please list the other RLIS goals and objectives. 



16) Do you communicate with the districts in your state specifically about the RLIS 
Program? 

No — > Go to question 19 
Yes —y Go to question 17 
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17) On what topics do you communicate? (Mark all that apply) 

Eligibility for RLIS funds 
RLIS application process 
Allowable costs 

Assistance in developing RLIS activities 
Other (please specify) 

If you selected other, please specify: 



18) On average, how often do you communicate with each district in your state about the 
RLIS Program? 

Rarely (1-2 times per year) 

Occasionally (Monthly or every other month) 

Frequently (More than once a month) 

19) Do you provide technical assistance or guidance to districts receiving RLIS funds? 

No — > Go to question 21 
Yes ^ Go to question 20 

20) What types of technical assistance or guidance do you provide to districts receiving 
RLIS funds? (Mark all that apply) 

Handbook or guidelines in handbook on appropriate use of RLIS funds 
Conference or workshop presentations on RLIS program 
Information provided on RLIS program on state Web site 
Checklist on appropriate use of funds 

Workshops or conference sessions devoted to receipt of RLIS funds 
District budget review of RLIS fund appropriations 
Other (please specify) 

If you selected other, please specify: 

21) How do you monitor districts’ use of RLIS funds? 

(Mark all that apply) 

Budget review 

On-site monitoring or audits 

Progress reports 

Required self-assessments or self-evaluations by districts 

Monitoring of RLIS is incorporated into general monitoring of funds for districts in 
need of improvement 
Other (please specify) 

If you selected other, please specify: 
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22) What happens when an RLIS-funded district does not make progress towards its 
goals? 

On-site monitoring or audits 

Technical Assistance for using RLIS funds to address specific problems 
Special monitoring or supervision of RLIS expenditures 
Other (please specify) 

If you selected other, please specify; 

23) Have you ever evaluated your RLIS Program? 

No — > Go to question 27 
Yes —>■ Go to question 24 

24) Did you produce a report or document based on the evaluation? 

No — > Go to question 27 
Yes ^ Go to question 25 

25) Is the report available on the Web? 

No [NOTE: BPA will follow up with you to get a hard-copy] —>■ Go to question 27 
Yes —y Go to question 26 

26) Please list the URL where the report can accessed. 



27) Do you have any suggestions for improvement of the RLIS program? 

No — > Go to question 29 
Yes —y Go to question 28 

28) What suggestions do you have for improvement of the RLIS program? 



29) Is there anything else you think we should know about your state’s RLIS program? 

No ^ End of survey 
Yes —y Go to question 30 

30) What else should we know about your state's RLIS program? 



Thank you again for taking our survey and participating in this evaluation! 



Appendix E 



124 






125 




APPENDIX F: RLIS DISTRICT COORDINATOR SURVEY 



According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of 
information unless such collection displays a valid OMB control number. The valid OMB control number for this 
information collection is xxxx-xxxx. The time required to complete this information collection is estimated to 
average 20 minutes per response, including the time to review instructions, search existing data resources, gather the 
data needed, and complete and review the information collection. If you have any comments concerning the 
accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of 
Education, Washington, D.C. 20202-4651. If you have comments or concerns regarding the status of your individual 
submission of this form, write directly to: Erica Lee, U.S. Department of Education, Office of Planning, Evaluation 
and Policy Development (OPEPD), Policy and Program Studies Service, 400 Maryland Avenue, SW, Washington, 
D.C. 20202. 

Responses to this data collection will be used only for statistical purposes. The reports prepared for this study will 
summarize findings across the sample and will not associate responses with a specific district/state or individual. We 
will not provide information that identifies you or your district/state to anyone outside the study team, except as 
required by law. 

Your cooperation in completing this survey will help to make the results of this evaluation comprehensive, reliable, 
and timely. 



1) In addition to the primary goal of making Adequate Yearly Progress, to what extent 
are the following a focus for your district's use of RLIS funds? 





H 


Minimal 

focus 


Moderate 

focus 


Major 

focus 


a. Increasing student achievement in a particular subject area 










b. Reducing the high school dropout rate 










c. Improving the quality of instruction 










d. Improving the ability of English language learners to achieve 
proficiency in English and reach high academic standards 










e. Ensuring that all students will be educated in learning environments 
that are safe, drug free, and conducive to learning 










f. Addressing issues specific to rural location, e.g., retaining teachers, 
providing distance learning opportunities, etc. 











2) In addition to the primary goal of making Adequate Yearly Progress and those listed 
above, are RLIS funds used to meet any other goals? 

No — > Go to question 4 
Yes ^ Go to question 3 

3) Please list any additional goals. 
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4) To what extent are the following activities a focus for your district's use of RLIS 
funds? 





H 


Minimal 

focus 


Moderate 

focus 


Major 

focus 


a. Teacher recruitment and retention, including the use of signing 
bonuses and other financial incentives 










b. Teacher professional development 










c. Educational technology, including software and hardware 










d. Parental involvement activities 










e. Activities authorized under the Safe and Drug-Free Schools Program 










f. Activities authorized under Title I, Part A 










g. Language instruction for ELL/LEP students 











5) Are there any other activities that are a focus for your district's use of RLIS funds? 

No — > Go to question 7 
Yes ^ Go to question 6 

6) Please list any other activities that are a focus for your district's use of RLIS funds. 



7) To what extent do you feel your district has made progress toward its RLIS goals and 
objectives, including the goal of making Adequate Yearly Progress? 

No progress —>■ Go to question 1 1 
Minimal progress — > Go to question 8 
Moderate progress — > Go to question 8 

Goals and objectives have been accomplished — > Go to question 8 

8) To what extent has the RLIS funding contributed to your district's progress towards 
its RLIS goals and objectives? 





No 

contribution 


Minimal 

contribution 


Moderate 

contribution 


Major 

contribution 


a. Making Adequate Yearly Progress 










b. Increasing student achievement in a particular 
subject area 










c. Reducing the high school dropout rate 










d. Improving the quality of instruction 










e. Improving the ability of English language learners 
to achieve proficiency in English and reach high 
academic standards 










f. Ensuring that all students will be educated in 
learning environments that are safe, drug free, and 
conducive to learning 










g. Addressing issues specific to rural location, e.g., 
retaining teachers, providing distance learning 
opportunities, etc. 
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9) Has RLIS funding contributed to your district’s progress towards any other goals and 
objectives? 



No — > Go to question 1 1 
Yes —>■ Go to question 10 



10) Please list the other RLIS goals and objectives. 



11) Do you communicate with your State Education Agency specifically about the RLIS 
Program? 

No — > Go to question 14 
Yes —>■ Go to question 12 

12) On what topics do you communicate? (Mark all that apply) 

Eligibility for RLIS funds 
RLIS application process 
Allowable costs 

Planning/developing RLIS -funded activities 
Other (please specify) 

If you selected other, please specify: 

13) How often do you communicate with your State Education Agency about the RLIS 
Program? 

Rarely (1-2 times per year) 

Occasionally (Monthly or every other month) 

Lrequently (More than once a month) 

14) To what extent have you received technical assistance or guidance from your State 
Education Agency for the RLIS Program? 

Not at all —>■ Go to question 20 
To a minimal extent — > Go to question 15 
To a moderate extent —>■ Go to question 15 
To a great extent — > Go to question 15 

15) How did you use the information and/or technical assistance provided by your State 
Education Agency? (Mark all that apply) 

To complete our application or comprehensive plan for funds 
To learn about/check on appropriate use of funds 
To identify areas needing attention 
To come up with new ideas for spending RLIS funds 
Other (please specify) 

If you selected other, please specify: 
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16) How helpful have you found the following forms of state technical assistance? 





Did not 
receive 


Received, but not 
helpful 


Minimally 

helpful 


Moderately 

helpful 


Very 

helpful 


a. Handbook or guidelines in handbook on 
appropriate use of RLIS funds 












b. Conference or workshop presentations on 
RLIS program 












c. Information provided on RLIS program on 
state Web site 












d. Checklist on appropriate use of funds 












e. Workshops or conference sessions devoted 
to receipt of RLIS funds 












f District budget review of RLIS fund 
appropriations 













17) Are there any other forms of state technical assistance that you have found to he 
helpful? 

No — > Go to question 19 
Yes Go to question 18 

18) Please list the other forms of state technical assistance that you have found to he 

helpful. 



19) Do you have any suggestions for other types of technical assistance or guidance that 
would he useful for you in your RLIS Program? 

No — > Go to question 21 
Yes — > Go to question 20 

20) What kinds of technical assistance or guidance would you like to receive from your 

State Education Agency for the RLIS Program? Please explain. 



21) Do you have any suggestions for improvement of the RLIS Program? 

No — > Go to question 23 
Yes Go to question 22 

22) What suggestions do you have for improvement of the RLIS program? Please explain. 
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23) Is there anything else you think we should know about your district’s RLIS program? 

No ^ End of survey 
Yes —y Go to question 24 

24) What else should we know about your district's RLIS program? Please explain. 

Thank you for taking our survey and participating in this evaluation! 
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The Department of Education’s mission is to promote student achievement 
and preparation for global competitiveness by fostering educational excellence 

and ensuring equal access. 



www.ed.gov 




